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PREFACE 


This  paper  documents  work  performed  by  the  Institute  for  Defense  Analyses 
(IDA)  in  partial  fulfillment  of  the  task  titled  “Systems  Engineering  in  the  New 
Acquisition  Process.”  The  Office  of  the  Under  Secretary  of  Defense  (Acquisition, 
Technology,  and  Logistics),  Office  of  Enterprise  Development  in  the  Office  of  Systems 
Engineering  (SE)  sponsored  this  work. 

This  paper  presents  IDA’s  work  in  examining  systems  engineering-related 
education  and  training  for  the  acquisition  workforce  to  enhance  the  implementation  of 
systems  engineering — specifically,  work  done  in  reviewing  and  recommending  revisions 
to  the  Defense  Acquisition  University  (DAU)  courses  for  the  Systems  Planning, 
Research,  Development,  and  Engineering/Systems  Engineering  (SPRDE/SE)  career  path. 

The  authors  wish  to  thank  the  reviewers,  Karen  Tyson  and  Lance  Hancock  of  IDA 
and  Ann  Marie  Choephel  from  OUSD(AT&L)  DS/SE/ED  Quadelta.  In  addition,  the 
authors  wish  to  thank  the  editor,  Shelley  Smith  of  IDA. 
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SUMMARY 


In  FY2004,  the  Systems  Engineering  office  asked  the  Institute  for  Defense 
Analyses  (IDA)  to  participate  in  a  review  of  the  systems  engineering-related  education 
and  training  offered  by  the  Defense  Acquisition  University  (DAU)  and  to  detennine  ways 
that  it  could  be  revised  or  updated  to  achieve  better  implementation  of  systems 
engineering  in  acquisition  programs.  Most  of  the  work  centered  on  the  courses  offered 
within  the  Systems  Engineering  (SE)  career  path  of  the  Systems  Planning,  Research, 
Development  and  Engineering  (SPRDE)  career  field.  The  purpose  of  this  paper  is  to 
document  IDA’s  participation  in  this  effort. 

At  the  time  of  this  review,  DoD  was  in  the  midst  of  a  department-wide 
transformation.  The  DoD  5000  series  had  been  updated,  a  new  “requirements” 
development  process  called  JCIDS  (Joint  Capabilities  Integration  Development  System) 
was  adopted,  and  systems  engineering  was  undergoing  revitalization.  All  of  these  updates 
and  initiatives  needed  to  be  reflected  in  SPRDE/SE  education  and  training  elements  and 
materials  such  as  the  SPRDE/SE  position  category  description,  certification 
requirements,  guidance  regarding  certification  course  content  (referred  to  in  the  paper  as 
“competencies”),  certification  course  content,  and  other  education  and  training-related 
resources. 

The  review  was  a  collaborative  effort  involving  the  Systems  Engineering  office, 
DAU,  and  other  members  of  the  SPRDE/SE  Functional  Integrated  Process  Team 
(FIPT) — the  team  responsible  for  managing  the  SPRDE/SE  career  path.  The  SPRDE/SE 
FIPT,  led  by  the  Deputy  Director  of  Enterprise  Development  in  the  Systems  Engineering 
Office,  was  ultimately  responsible  for  the  review  and  for  providing  guidance  to  DAU 
regarding  needed  revisions  and  updates  to  the  certification  course  material  and  other 
education  and  training  related  resources.  DAU,  as  the  developer  of  education  and  training 
resources,  was  responsible  for  updating  and  revising  the  education  and  training  resources 
based  on  the  FIPT’s  guidance. 

IDA  supported  this  effort  by  proposing  an  overall  approach  to  reviewing 
SPRDE/SE  education  and  training  requirements,  guidance  and  materials,  and  by  using 
that  approach  to  develop  a  list  of  duties  and  tasks  that  today’s  SPRDE/SE  professionals 
perform.  These  duties  and  tasks  were  intended  to  serve  as  guidance  from  the  FIPT  to 
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DAU  SPRDE/SE  curriculum  developers.  The  review  approach  IDA  proposed  and  used 
was  a  combined  top-down  and  bottom-up  approach.  The  top-down  review  consisted  of 
reviewing  current  DoD  and  Systems  Engineering  policy  guidance  and  initiatives,  systems 
engineering  processes  and  activities,  and  best  practices.  The  bottom-up  review,  on  the 
other  hand,  was  a  review  of  current  SPRDE/SE  certification  course  content,  and  existing 
and  past  guidance  for  those  courses.  Combining  these  two  approaches  allowed  IDA  to 
revise  and  build  upon  existing  guidance  while  incorporating  new  elements  relevant  to 
today’s  DoD  acquisition  environment.  The  products  of  the  review,  including  IDA’s  main 
product,  the  SPRDE/SE  Duties  and  Tasks  (found  in  appendix  K)  are  included  among  the 
appendixes  to  this  paper. 

Included  below  are  process-related  recommendations  resulting  from  IDA’s 
experience  in  the  review: 

Recommendation  1:  Conduct  a  more  thorough  requirements  analysis  of 
SPRDE/SE  acquisition  personnel  and  their  needs  for  SPRDE/SE  training.  In  the  course  of 
the  review,  we  did  not  have  the  time  to  do  as  much  of  the  requirements  development 
process,  one  of  the  most  important  processes  in  systems  engineering,  as  we  would  have 
liked.  Having  a  better  sense  of  the  requirements  would  have  led  to  more  complete  criteria 
for  reviewing  the  courses.  It  would  be  interesting,  for  example,  to  examine  the  areas 
where  systems  engineers  have  the  most  trouble  in  programs,  and  to  emphasize  those  areas 
in  the  SPRDE/SE  education  and  training.  We  attempted  to  do  this  through  previous 
studies  but  did  not  have  the  real-time  data  that  we  needed.  An  analysis  like  this  might 
also  be  useful  in  determining  whether  the  certification  courses  are  the  resources  that  need 
update  and  investment  or  if  courses  that  are  more  widely  available  on-demand  (such  as 
Continuous  Learning  Modules  (CLMs))  might  be  a  better  approach.  The  assessment  data 
needs  to  feed  into  the  FIPT  process. 

Recommendation  2:  DAU  should  clarify  and  standardize  instructional 
tenninology.  If,  rather  than  mandating  use  of  a  specific  process,  DAU  wants  to  ensure 
that  each  of  the  FIPTs  can  be  flexible  in  their  management  of  the  career  fields  and  paths, 
it  would  be  helpful  if  DAU  defined  its  terminology  generically  enough  to  allow  for  this 
variation.  While  the  instructional  terminology  definitions  in  Chapter  II  are  helpful  as  a 
starting  point  in  understanding  the  education  and  training  review  process,  the  use  of  the 
tenninology  is  confusing.  Although  the  FIPT  charter  indicates  that  “Competencies” 
should  be  developed/reviewed,  career  fields/paths  develop  Learning  Outcomes  (LOs), 
duties  and  tasks,  and  the  like  instead  of  competencies,  hence  our  recommendation. 
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Recommendation  3:  Define  requirements  carefully  up  front,  and  rigorously  follow 
systems  engineering  processes.  Our  process,  like  so  many  programs’  processes,  suffered 
from  requirements  changes  along  the  way.  For  example,  we  started  out  with  a  goal  to 
create  duties  and  tasks,  the  requirement  changed  to  develop  Performance  Objectives,  and 
the  group  finally  settled  on  LOs.  One  example  where  we  could  have  used  stronger 
systems  engineering  processes  was  in  configuration  management — specifically  tracking 
decisions  made  throughout  the  review  process.  Agendas  and  follow-up  minutes  are 
always  developed  for  the  SPRDE/SE  FIPT,  but  perhaps  this  practice  would  be  useful  for 
the  smaller  meetings  as  well.  Our  experience  indicated  that  decisions  made  at  one 
meeting  that  necessitated  follow-up  actions  were  sometimes  negated  in  future  meetings. 
Another  example  relates  to  the  configuration  control  of  the  review  documents.  The 
education  and  training  review  was  a  collaborative  effort  with  participation  from  many 
people  and  organizations.  Given  the  number  of  people  and  organizations  editing 
documents,  we  may  need  to  use  more  sophisticated  ways  of  tracking  documents’  versions 
and  configuration.  Perhaps  we  could  make  better  use  of  the  SE  CoP  workspace  provided 
to  the  group  by  DAU  for  this  purpose. 
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I.  INTRODUCTION 


In  FY2004,  the  Systems  Engineering  office  asked  the  Institute  for  Defense 
Analyses  (IDA)  to  conduct  a  review  of  the  systems  engineering-related  education  and 
training  offered  by  the  Defense  Acquisition  University  (DAU),  and  to  determine  ways 
that  it  could  be  revised  or  updated  to  achieve  better  implementation  of  systems 
engineering  in  acquisition  programs.  The  DAU  education  and  training  resources  are 
periodically  reviewed  to  ensure  that  they  are  meeting  the  needs  of  their  students  and  DoD 
the  acquisition  community.  At  the  time  of  this  review,  many  activities  were  occurring  to 
revitalize  systems  engineering  within  the  Department  of  Defense  (DoD),  including  the 
development  of  new  systems  engineering  policy. 

The  purpose  of  this  paper  is  to  document  IDA’s  participation  in  the  review, 
particularly  of  the  Systems,  Planning,  Research,  Development  and  Engineering/Systems 
Engineering  (SPRDE/SE)  career  path.  The  first  chapter  provides  a  background  for  the 
review,  summarizing  recent  changes  to  the  DoD  acquisition  policy,  including  the  systems 
engineering  revitalization  efforts.  Chapter  II  then  describes  the  Defense  Acquisition 
Workforce  Improvement  Act  (DAWIA)  process  which  is  the  context  for  the  education 
and  training  reviews,  and  provides  an  overview  of  how  IDA  worked  within  that  process. 
Subsequent  chapters  describe  the  implementation  of  the  review,  as  illustrated  in  Figure 
1-1.  Chapter  III  discusses  changes  made  to  SPRDE/SE  career  path  requirements,  Chapter 
IV  covers  modification  and  review  of  the  SPRDE/SE  training  materials,  and  Chapter  V 
describes  the  process  of  developing  SPRDE/SE  “competencies,”  in  the  form  of  duties  and 
tasks  as  a  guide  for  SPRDE/SE  certification  course  content.  Each  of  these  three  chapters 
documents  the  education  and  training  requirements/courses/competencies  before  the 
review  as  well  as  the  recommended  changes  resulting  from  the  review.  The  paper 
concludes  with  next  steps  and  recommendations  presented  in  Chapter  VI  and  is  followed 
by  several  appendixes  containing  supporting  infonnation.  The  last  appendix,  P,  lists  and 
defines  acronyms  used  in  the  report. 


1-1 


Figure  1-1.  Structure  of  the  Paper 

A.  DEPARTMENT  OF  DEFENSE  POLICY  CHANGES 


At  the  time  of  this  review,  DoD  was  in  the  midst  of  a  department-wide 
“transformation”  into  a  more  adaptive,  capabilities-based  organization.  In  the 
Transformation  Planning  Guidance,  Secretary  Rumsfeld  described  the  outcome  of  the 
transformation  as  “fundamentally  joint,  network-centric,  distributed  forces  capable  of 
rapid  decision  superiority  and  massed  effects  across  the  battlespace.”1 

There  are  three  parts  to  DoD’s  overall  strategy  for  transformation: 

1)  Transformed  culture  through  innovative  leadership 

2)  Transformed  Processes — Risk  adjudication  using  future  operating  concepts 

—  Balance  current  requirements  with  future  requirements 

—  Reformed  Capabilities-identification  process:  Joint  Capabilities 
Integration  and  Development  System  (JCIDS) 

—  Transformed  strategic  analysis 


i 


2 


DoD  Transformation  Guidance,  April  2003,  p.l 
Ibid.,  p.  8 
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3)  Transformed  Capabilities  through  force  transformation 
—  Strengthening  j  oint  operations 
—  Exploiting  US  intelligence  advances 
—  Experimenting  with  new  warfighting  concepts 
—  Developing  transformational  capabilities 

The  Transformation  process  is  changing  “the  way  we  fight,  the  way  we  do 
business  within  the  Department  and  the  way  we  work  with  our  interagency  and 
multinational  partners.”  Key  to  changing  the  way  DoD  does  business  is  reforming  the 
acquisition  process.  Transformation  seeks  to  shorten  the  acquisition  cycle  time  and  use  a 
new  “resource  allocation  process  built  around  joint  operating  concepts”  by  linking 
acquisition  strategies  to  future  joint  concepts.* 4  The  next  two  sections  of  this  chapter 
discuss  these  topics  in  further  detail. 

1.  Department  of  Defense  5000  Acquisition  Series 

The  DoD  5000  Series  documents,  DoD  Directive  (DoDD)  5000.1  and  DoD 
Instruction  (DoDI)  5000.2,  were  updated  in  May  2003.  The  Regulation,  DoD  5000. 2-R, 
was  canceled  and  will  be  replaced  by  the  Acquisition  Guide,  one  chapter  of  which  is 
devoted  to  systems  engineering.  According  to  the  DoD  5000  Resource  Web  site,  the  new 
version  of  the  DoD  5000  series  documents  differ  from  the  old  one  in  that  the  new 
series — 

•  Encourages  innovation  and  flexibility 

•  Pennits  greater  judgment  in  the  employment  of  acquisition  principles 

•  Focuses  on  outcomes  vice  processes 

•  Empowers  program  managers  to  use  the  acquisition  system  vice  being 
hampered  by  over-regulation  5 

A  Dayton  Aerospace  paper  summarized  more  specific  changes  to  the  5000  series.6 
One  is  that  evolutionary  acquisition  and  spiral  development  are  the  preferred  approaches 
to  system  development.  Both  of  these  are  systems  engineering  models.  The  2003  DoD 
5000  also  introduces  the  new  Joint  Capabilities  Integration  Development  System,  which 


Ibid.,  p.  6 

4  Ibid.,  p.  7 

DoD  5000  Resource  Web  site,  Tutorial,  http://dod5000.dau.mil/Tutorial/index.htm  (accessed 
16  August  2004) 

“New”  DoD  5000  Series  Implications,  Executive  Summary,  http://www.davtonaero.com/documents/ 
5000  Series  Implications  5-20-03.pdf  (accessed  16  August  2004)  p.  1 
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replaces  the  old  Requirements  Generation  System.  JCIDS,  as  it  is  called,  is  described  in 
more  detail  in  the  next  section.  Changes  to  the  acquisition  phases  include  the  following: 

•  Concept  Refinement  and  Technology  Development  are  now  two  distinct 
phases 

•  Milestone  A  is  now  at  the  end  of  Concept  Refinement 

•  A  Design  Readiness  Review  is  required  to  enter  the  System  Demonstration 
phase 

•  Milestone  A  is  required  to  enter  Technology  Development 

The  preference  for  spiral  development  and  evolutionary  acquisition,  in  particular, 
is  a  good  example  to  illustrate  the  importance  of  systems  engineering  education.  These 
models  are  not  widely  understood  in  the  community.  The  topics  should  be  covered 
carefully  in  systems  engineering  education  and  training  materials  to  ensure  that  the 
acquisition  workforce  has  the  background  it  needs  to  understand  whether  these 
approaches  are  the  right  ones  for  their  particular  systems/programs  and  to  provide  the 
systems  engineers  with  background  they  need  to  employ  these  approaches. 

2.  Chairman  of  the  Joint  Chiefs  of  Staff  3170.01  Instruction  and  Manual 

DoD  Systems  Engineers  now  must  think  in  terms  of  “capabilities”  rather  than 
requirements.  Understanding  the  requirements  (or  capabilities,  as  the  case  may  be)  has 
long  been  considered  one  of  the  most  important  parts  of  systems  engineering.  The  new 
Joint  Capabilities  Integration  and  Development  System  (JCIDS)  is  using  good  systems 
engineering  approaches  early  in  the  acquisition  cycle,  to  provide  joint,  integrated 
solutions  (materiel  and  non-materiel)  to  the  warfighter.  The  Chairman  of  the  Joint  Chiefs 
of  Staff  (CJCS)  3170.01  Instruction  and  Manual  describes  the  policies,  procedures,  and 
guidelines  of  JCIDS.  Using  joint  concepts  and  integrated  architectures,  JCIDS  identifies 
and  prioritizes  capability  gaps  and  solutions  to  fill  those  gaps.  Through  this  process,  DoD 
intends  to  eliminate  redundancy  in  new  capability  development. 

There  are  three  main  analyses  performed  in  JCIDS:  the  Functional  Area  Analysis 
(FAA),  the  Functional  Needs  Analysis  (FNA),  and  the  Functional  Solution  Analysis 
(FSA).  The  FAA  identifies  operational  tasks,  conditions  and  standards  needed  to  achieve 
military  objectives.  The  FNA  then  assesses  the  capability  of  current,  joint  capabilities  to 
accomplish  the  tasks  identified  in  the  FAA.  Resulting  from  the  FNA  is  a  list  of  capability 
gaps  and  redundancies,  along  with  timeframes  when  the  gaps  need  to  be  filled.  Finally, 
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the  FSA  is  an  assessment  of  all  potential  Doctrine,  Organization,  Training,  Materiel, 
Leadership,  Personnel,  and  Facilities  (DOTMLPF)  approaches  to  address  the  gaps.7 

Several  new  documents  are  required  as  a  part  of  the  JCIDS  process.  These 

8 

documents  are  summarized  below. 

•  Initial  Capabilities  Document  (ICD) —  replaces  a  Mission  Needs  Statement 
(MNS) 

—  Identifies  a  capability  gap 
—  Describes  evaluation  of  DOTMLFP  approaches 
—  Supports  the  Analysis  of  Alternatives  (AoA)  in  Concept  Refinement 
—  Not  updated  once  approved 

•  Capability  Development  Document  (CDD) — replaces  the  Operational 
Requirements  Document  (ORD)  at  Milestone  B 

—  CDDs  are  system  specific  and  apply  to  a  single  increment  of  an 
evolutionary  acquisition  approach 

—  Results  from  Technology  Development  phase  of  the  acquisition  cycle 
—  Updated  for  subsequent  increments 

•  Capability  Production  Document  (CPD) — replaces  the  ORD  at  Milestone  C 
—  Identifies  production  attributes  for  a  single  increment  of  a  program 
—  Prepared  during  System  Development  and  Demonstration  (SDD) 

—  Rewritten  for  each  increment  in  an  evolutionary  program. 

•  Capstone  Requirements  Document  (CRD) — unchanged  from  the 

Requirements  Generation  system 

—  Describes  overarching  thresholds/goals  and  standards  in  functional  areas 

—  Developed  only  at  Joint  Requirements  Oversight  Council  (JROC) 
direction. 

The  CRD  will  eventually  be  replaced  by  integrated  architectures  as  they  are 
developed  and  implemented. 

B.  SYSTEMS  ENGINEERING  POLICY  CHANGES 

Amidst  the  overall  Transformation  in  DoD,  Systems  Engineering  Revitalization 
efforts  were  taking  place  in  the  Systems  Engineering  office.  The  Honorable  Michael 
Wynne,  Acting  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics 


CJCSM  3170.01A,  12  March  2003,  p.  A-l 

Descriptions  of  the  JCIDS  documentation  is  taken  from  DAU’s  “DoD  Business  Transformation” 
briefing,  http://dod5000.dau.mil/DQCS/257, 1, DoD  Business  Transformation  Meeting  the  Security 

Challenges  of  the  2 1ST  century  (accessed  26  August  2004) 
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(USD  (AT&L)),  and  Dr.  Glenn  Lamartin,  Director  of  Defense  Systems,  both  issued 
policy  memorandums:  Mr.  Wynne’s  memo  focused  on  systems  engineering  policy,  and 
Dr.  Lamartin’s  on  SEP  guidance.  Meanwhile,  the  Enterprise  Development  office 
developed  vee-like9  process  models  to  help  explain  how  systems  engineering  should  be 
applied  throughout  the  acquisition  cycle  and  compiled  the  Systems  Engineering  chapter 
(Chapter  4)  of  the  Acquisition  Guidebook. 

1.  Systems  Engineering  Policy  Memorandum 

Mr.  Wynne  issued  new  systems  engineering  policy  in  a  memorandum  titled 
“Policy  for  Systems  Engineering  in  DoD”  to  help  “drive  good  systems  engineering 
processes  and  practice  back  into  the  way  we  do  business”10  Signed  on  20  February  2004, 
the  memo  established  policy  that  took  effect  immediately  and  that  was  to  be  included  in 
the  next  DoD  5000  series  acquisition  documents.  The  new  systems  engineering  policy  is 
as  follows: 

Systems  Engineering  (SE).  All  programs  responding  to  a  capabilities  or 
requirements  document,  regardless  of  acquisition  category,  shall  apply  a 
robust  SE  approach  that  balances  total  system  performance  and  total 
ownership  costs  with  the  family  of  systems,  systems-of-systems  context. 
Programs  shall  develop  a  Systems  Engineering  Plan  (SEP)  for  Milestone 
Decision  Authority  (MDA)  approval  in  conjunction  with  each  Milestone 
review,  and  integrated  with  the  Acquisition  Strategy.  This  plan  shall 
describe  the  program’s  overall  technical  approach,  including  processes, 
resources,  metrics,  and  applicable  performance  incentives.  It  shall  also 
detail  the  timing,  conduct  and  success  criteria  of  technical  reviews.  11 

The  memo  lays  out  responsibilities  for  the  Director  of  Defense  Systems  to  support 
the  policy — one  of  which  is  to  “identify  the  requirement  for  a  SEP  in  DODI  5000.2,”  and 
to  provide  specific,  yet  tailorable,  guidance  on  that  SEP  in  the  Acquisition  Guidebook. 
The  Director  of  Defense  Systems  also  shall  review  and  provide  recommendations 
regarding  each  program’s  SEP  (for  those  programs  where  Mr.  Wynne  is  the  MDA  in 
preparation  for  Defense  Acquisition  Board  (DAB)  Milestone  Reviews  and  other 
acquisition  reviews).  A  third  responsibility  is  to  establish  a  senior-level  SE  forum  to  help 
institutionalize  systems  engineering  discipline  in  DoD.  The  responsibility  most  directly 


9 


10 


11 


information  about  the  vee  model  is  available  at 
http://acc.dau.mil/simplify/ev_en.php?ID=6260_201&ID2=DO_TOPIC 

Memorandum  from  Michael  Wynne,  “Policy  for  Systems  Engineering  in  DoD,”  issued  20  February 

2004 

Ibid. 
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related  to  the  education  and  training  review  documented  in  this  paper  is  that  the  Director 
of  Defense  Systems  shall  “assess  the  adequacy  of  current  Department-level,  Systems 
Engineering  related  policies,  processes,  practices,  guidance,  tools,  and  education  and 
training .”  Mr.  Wynne’s  memo  is  included  in  Appendix  A  of  this  paper. 

2.  Lamartin  Systems  Engineering  Plan  Memo  and  Guidance 

In  response  to  the  policy  issued  by  the  acting  USD  (  AT&L),  Dr.  Lamartin,  the 
Director  of  Defense  Systems,  issued  interim  guidance  about  the  SEP  in  a  memorandum 
titled,  “Implementing  Systems  Engineering  Plans  in  DoD — Interim  Guidance.”  It  was 
signed  and  released  on  30  March  2004.  The  memo  states  that  the  SEP  “is  intended  to  be  a 
living  document,  tailored  to  the  program,  and  a  roadmap  that  supports  program 
management  by  defining  comprehensive  systems  engineering  activities,  addressing  both 
government  and  technical  activities  and  responsibilities.”  ~  There  is  no  set  format  for  the 

1 3 

SEP,  but  the  memo  highlights  the  following  areas  that  the  plan  must  address: 

•  Systems  engineering  processes  to  be  applied  to  the  program 

•  System’s  technical  baseline  approach 

•  Event  driven  timing,  conduct,  success  criteria  and  expected  products  of 
technical  reviews 

•  Integration  of  systems  engineering  into  the  program’s  integrated  product 
teams  (IPTs) 

The  SE  Office  included  some  additional  SEP  guidance  in  the  systems  engineering 
chapter  of  the  Acquisition  Guidebook  and  is  currently  creating  more  detailed  guidance 
including  a  proposed  SEP  outline.  Dr.  Lamartin’s  memorandum  is  attached  in 
Appendix  B. 

3.  Systems  Engineering  Revitalization  “Hooks”  and  Policy 

With  the  revitalization  of  systems  engineering,  the  Enterprise  Development  office 
developed  new  ideas  for  guidance,  which  it  referred  to  as  “hooks.”  The  hooks  represent 
the  office’s  current  thinking  on  where  systems  engineering  needs  to  go  and  may  indicate 
the  direction  for  new  guidance  coming  out.  Not  all  of  these  hooks  are  currently  policy, 
although  many  are,  such  as  the  requirement  for  a  SEP.  The  hooks  are  the  following — 

•  Develop  a  systems  engineering  strategy 


Dr.  Glenn  Lamartin,  “Implementing  Systems  Engineering  Plans  in  DoD — Interim  Guidance,” 
Memorandum,  30,  March  2004 
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•  Address  systems  engineering  (all  acquisition  phases,  not  just  SDD)  up  front 
and  early 

•  Consider  systems  engineering  in  total  life  cycle  systems  management 

•  Emphasize  in-service  systems  engineering 

•  Ensure  that  systems  engineering  spans  IPT  (not  located  in  an  “SE  IPT”) 

•  Acquisition  components  must  maintain  a  robust  systems  engineering 
organization 

•  Ensure  that  lead  engineer  is  jointly  accountable  to  Program  Manager  (PM) 
and  functional  leadership 

•  Assign  a  lead  systems  engineer  at  program  inception 

•  Establish  a  SEP  early  in  the  program  definition  and  update  it  continuously  as 
the  program  matures,  through  system  operations  and  support 

•  Include  in  the  SEP  description  of  systems  integration  on  the  program  IPTs, 
including  resources,  staffing,  management  metrics,  and  integration 
mechanisms 

•  Form  a  team  of  independent  subject  matter  experts  (SMEs)  and  program  team 
representatives  to  conduct  systems  engineering  technical  reviews 

•  Ensure  that  systems  engineering  technical  reviews  are  chaired  by  an 
independent  technical  authority 

•  Contractual  documents  address  and  require  systems  engineering  technical 
reviews 

•  Tailor  technical  reviews  in  conjunction  with  the  systems  engineering 
processes  to  fit  the  acquisition  strategy 

•  Approve  technical  baselines  at  technical  reviews 

•  Ensure  that  technical  reviews  are  event  driven  vs.  schedule  driven 

•  Develop  and  use  metrics  in  technical  reviews 

•  Evaluate  the  system’s  technical  basis  for  cost 

•  Develop  and  maintain  requirements  in  an  integrated  framework 

These  hooks  are  not  published,  and  the  list  is  evolving.  The  list  shown  above  was 
created  based  on  discussions  with  the  Enterprise  Development  office.  We  used  this  list  to 
support  our  review. 

4.  Integrated  Defense  Acquisition,  Technology  and  Logistics  Life  Cycle 
Management  Framework 

In  an  effort  to  provide  the  acquisition  community  with  more  direction  on  how  to 
conduct  systems  engineering  in  the  DoD  acquisition  phases,  the  Office  of  Enterprise 
Development  created  systems  engineering  “Vee”  process  models  corresponding  to  the 
acquisition  phases  (e.g.,  Concept  Refinement,  Technology  Development).  Although  these 
models  for  each  phase  are  in  the  shape  of  a  traditional  “Vee”  model,  they  differ  in  their 
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use  of  processes  within  the  model.  As  a  result,  we  call  them  “process-vees”  throughout 
this  paper.  During  the  development  of  these  process-vees,  DAU  was  in  the  midst  of 
revising  a  wall  chart  reference  and  teaching  tool  titled  “Integrated  Defense  (AT&L)  Life 
Cycle  Management  Framework.”  DAU  develops  such  wall  charts  periodically  as  a 
classroom  aid  for  DAU  students.  After  seeing  an  early  version  of  the  wall  chart,  the 
Enterprise  Development  Office  saw  the  chart  as  a  potential  vehicle  to  communicate  and 
teach  the  process-vee  models  and  requested  that  DAU  incorporate  them.  DAU  agreed  and 
the  wall  chart  now  includes  the  models  mapped  to  the  DoD  5000  acquisition  phases.  The 
Integrated  Defense  AT&L  Lifecycle  Management  Framework  also  shows  other 
information  such  as  the  inputs  and  outputs  of  each  of  the  acquisition  phases  and  the 
timing  of  technical  reviews  with  respect  to  the  process-vee  models. 

5.  Acquisition  Guidebook 

The  Enterprise  Development  office  also  coordinated  the  development  of  the 
Systems  Engineering  chapter  (Chapter  4)  of  the  Acquisition  Guidebook. 14  The  Systems 
Engineering  chapter  of  the  Guidebook  includes  the  following  sections: 

•  Systems  Engineering  in  DoD  Acquisition 

•  Systems  Engineering  Processes:  How  Systems  Engineering  Is  Implemented 

•  Key  Systems  Engineering  Activities  in  the  System  Life  Cycle 

•  Systems  Engineering  Decisions:  Important  Design  Considerations 

•  Systems  Engineering  Execution:  Key  Systems  Engineering  Tools  and 
Techniques 

•  Systems  Engineering  Resources 

The  chapter  begins  with  a  section  titled  “Systems  Engineering  in  DoD 
Acquisition  Systems,”  which  describes  how  systems  engineering  is  related  to  DoD 
acquisition.  The  next  section,  “Systems  Engineering  Processes:  How  Systems 
Engineering  Is  Implemented,”  summarizes  the  systems  engineering  technical 
management  processes  and  technical  processes  and  the  use  of  systems  engineering 
standards  and  models.  “Key  Systems  Engineering  Activities  in  the  Systems  Life  Cycle” 
then  explains  how  the  process-vees  from  the  Defense  Acquisition  Framework  fit  into  the 
DoD  acquisition  cycle,  including  descriptions  and  timing  (event-based)  of  technical 
reviews.  “Systems  Engineering  Decisions:  Important  Design  Considerations”  includes  a 
list  of  important  design  considerations  that  systems  engineers  must  take  into  account  to 
develop  an  integrated  set  of  requirements.  Tools,  such  as  the  SEP,  required  and 


The  Guidebook  is  available  at  http://akss.dau.mil/DAG/  (accessed  1  December  2004) 
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elaborated  upon  by  the  Wynne  and  Lamartin  memos,  are  described  in  “Systems 
Engineering  Execution:  Key  Systems  Engineering  Tools  and  Techniques.”  Finally, 
“Systems  Engineering  Resources”  includes  links  to  systems  engineering  reference 
materials  from  DoD,  academia,  and  industry. 
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II.  SPRDE/SE  REVIEW  CONTEXT  AND  PROCESS 


IDA  conducted  its  review  of  systems  engineering-related  education  and  training 
within  the  context  of  the  Defense  Acquisition  Workforce  Improvement  Act  (DAWIA) 
designated  career  fields/paths  and  the  general  process  by  which  these  career  fields/paths 
are  managed.  This  chapter  describes  IDA’s  review  of  DAU’s  systems  engineering-related 
education  and  training  material  in  conjunction  with  the  activities  of  the  SPRDE/SE 
Functional  Integrated  Product  Team  (FIPT). 

A.  DEFENSE  ACQUISITION  WORKFORCE  IMPROVEMENT  ACT 

The  DAWIA  was  signed  into  law  in  November  1991  after  reviews  of  the  DoD 
acquisition  workforce  found  it  to  be  "undertrained,  underpaid,  and  inexperienced." 
DAWIA  was  enacted  through  an  FY  1991  Defense  Authorization  Bill  in  order  to  bring 
more  professionalism  to  acquisition  personnel.17  The  act  required  the  Secretary  of 
Defense,  acting  through  the  Under  Secretary  of  Defense  (Acquisition,  Technology,  and 
Logistics),  to  establish  education,  training  and  experience  requirements  for  the  civilian 
and  military  acquisition  workforce.  These  requirements  are  documented  in  DoD  5000.52- 
M,  "Career  Development  Program  for  Acquisition  Personnel.”16  DoD  5000. 52-M  states 
that  the  requirements  will  lead  to  improvements  in  the  workforce  by: 17 

a.  Developing,  on  a  long-term  basis,  a  highly  qualified  diverse  workforce 
capable  of  performing  current  and  future  DoD  acquisition  functions. 

b.  Meeting  current  and  future  DoD  needs  for  acquisition  personnel  and 
providing  capable  replacements  for  senior  acquisition  positions  on  a 
planned  and  systematic  basis. 

c.  Increasing  the  proficiency  of  DoD  acquisition  personnel  in  their 
present  positions  and  providing  guidance  and  opportunities  for 
broadening  experiences  and  progression  commensurate  with  their 
abilities 


“The  Defense  Acquisition  University  From  Consortium  to  Consolidation,”  Program  Manager,  May 
2001,  Kelly  Berta,  http://www.findarticles.eom/p/artieles/mi_m0KAA/is_3_30/ai_78399311 
(accessed  10  August  2004),  p.  1 

DAU’s  Acquisition  Community  Connection  Web  site:  http://acc.dau.mil/simplify/ 
ev.php?ID=10106  20 1  &1D2-DO  TOPIC  (accessed  15  September  2004) 


17 


Website  where  5000. 52-M  is  available  (accessed  10  August  2004),  p.I-1. 


d.  Ensuring  the  effective  use  of  training  and  education  resources 

The  act  also  mandated  the  establishment  of  a  Defense  Acquisition  University  to 

18 

coordinate  the  education  and  training  of  the  workforce.  DAU  and  its  relationship  to 
DAWIA  is  described  further  in  section  B  of  this  chapter. 

1.  Defense  Acquisition  Workforce  Improvement  Act  Functional  Areas  and  Career 
Fields/Paths 

DAWIA  identified  12  acquisition  career  fields  in  the  acquisition  workforce  for 
which  education,  training,  and  experience  requirements  were  described  in  5000. 52-M. 
“Career  field”  is  defined  as  “one  or  more  occupations  that  require  similar  knowledge  and 
skills.”  Career  fields  may  also  encompass  separate  “career  paths.”  For  example,  the 
SPRDE  career  field  has  two  career  paths:  Science  and  Technology  Manager  (STM)  and 
Systems  Engineering  (SE).  Career  fields  and  any  paths  they  encompass  are  referred  to 
generally  as  “career  fields/paths”  throughout  this  paper.  A  recent  list  of  the  DAWIA 
career  fields  (and  career  paths  for  SPRDE)  is  shown  in  the  second  column  of  Table  II- 1. 
These  fields  (and  associated  career  paths)  have  changed  some  since  DAWIA’s  inception. 
The  first  column  of  the  table  shows  the  functional  area  in  which  the  career  field  (and 
career  paths  for  SPRDE)  fits.  A  Functional  Advisor  oversees  the  management  of  those 
career  fields  or  paths  within  his  or  her  functional  areas. 

2.  Managing  the  Defense  Acquisition  Workforce  Improvement  Act  Career  Paths: 
Functional  Area  Charter 

Each  of  the  functional  areas  listed  in  the  first  column  of  Table  II- 1  is  managed  by 
a  Functional  Advisor.  FAs  are  senior-level  acquisition  executives  with  expertise  in  that 
particular  area.  An  Executive  Secretary  assists  the  FA  in  managing  the  career  fields/paths 
and  designates  a  Functional  Integrated  Product  Team  (FIPT).  The  Executive  Secretary 
and  FIPT  work  together  to  ensure  that  courses  are  up  to  date  and  relevant  to  the  needs  of 
the  workforce.  Taken  from  guidance  in  the  SPRDE/SE  Functional  Area  Charter,  the 
following  bullets  describe  the  responsibilities  of  the  FA  and  FIPT:  19 

•  Establish  and  review  the  DoD  criteria  for  designating  Position  Category 
Description(s)  (PCD)  and  career  path  certification  standards 

•  Annually  certify  experience,  education,  and  training  standards  and  position 
category  descriptions  as  current,  complete,  and  accurate 


Berta,  p.  1 

19  FIPT  Charter  from  AnnMarie  Choepel’s  minutes  of  the  11  December  SRPDE  FIPT  meeting.  Charter 
slide 
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Table  11-1.  DAWIA  Functional  Areas  and  Career  Fields/Paths 


Functional  Area 

Career  Fields/Paths 

Acquisition  Management 

Program  Management 

Auditing 

Auditing 

Business,  Cost  Estimating,  and  Financial 
Management  (BCEFM) 

BCEFM 

Facilities  Engineering 

Facilities  Engineering 

Information  Technology 

Information  Technology 

Logistics 

Life  Cycle  Logistics 

Procurement  &  Contracting/  Government 
Property 

Industrial/Contract  Property  Management 

Contracting 

Purchasing 

Science  and  Technology 

Systems  Planning,  Research,  Development  and 
Engineering  (SPRDE)/Science  and  Technology 
Manager  (STM) 

Technical  Management 

Production,  Quality  and  Manufacturing  (PQM) 

Systems  Planning,  Research,  Development  and 
Engineering  (SPRDE)/Systems  Engineering 
(SE) 

Test  and  Evaluation  (T&E) 

Source:  FAs  and  Career  Fields  were  taken  from  DAU  2004  Catalog,  pp.  5  and  19 
http://www.dau.mil/catalog/default.aspx  (accessed  10  August  2004).  Matchup  between  the  FAs  and  career  paths  was 
assisted  by  the  LMI  Report  http :// gray ity . lmi . or g/futurewf/  (accessed  10  August  2004)  section  5,  p.  18 


•  Annually  certify  content  and  quality  of  DAU  courses  as  current,  technically 
accurate,  and  consistent  with  DoD  acquisition  policies 

•  Recommend  initiatives  for  career  development  and  rotational  assignments 
between  various  DoD  Components  as  well  as  with  other  Government 
Agencies 

•  Promote  and  enable  multifunctional  career  paths  and  make  recommendations 
to  augment  existing  career  paths 

•  Oversee  education  and  training  requirements: 

—  Identify  competencies,  allocations,  quotas,  student  attendance,  course 
critiques,  priorities,  funding,  and  reports  under  DoDI  5000.58 

—  Make  recommendations  on  the  modifications,  establishment  or 
disestablishment  of  mandatory  courses 

—  Consider  continuous  learning  needs  and  resources  as  part  of  the  FA’s 
requirements  review  process 

—  Assist  the  DAU  program  Director  and  Course  Director(s)  as  necessary 
with  routine  updates  to  the  content  of  established  courses  to  maintain 
currency 
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•  Monitor  and  evaluate  the  effective  implementation  of  DoD  5000. 52-M  within 
the  functional  area 

The  FIPTs  commence  meetings  periodically  to  conduct  the  activities  stated  in  the 
charter  above.  When  a  career  field/path’s  FIPT  proposes  modifications  to  the  career 
field/path  elements,  those  modifications  are  submitted  through  the  Functional  Advisor  to 
the  Career  Management  Overarching  Integrated  Product  Team  (CMOAIPT).  Depending 
on  the  nature  of  the  modification,  the  CMOAIPT  either  makes  recommendations  to  the 
DAU  president  or  to  the  Director  Acquisition  Education,  Training  and  Career 
Development  (AET&CD).  Recommendations  regarding  requirements  for  new  courses  or 
major  changes  to  existing  courses  are  provided  to  the  DAU  president,  while 
recommendations  about  career  paths,  experience,  education,  and  training  standards  are 
sent  to  the  Director,  AET&CD.20 

B.  DEFENSE  ACQUISITION  UNIVERSITY 

The  Defense  Acquisition  University  (DAU)  develops  all  of  the  DAWIA  career 
field  and  career  path  certification  courses  as  well  as  other  defense  acquisition-related 
resources  covering  a  variety  of  topics.  DAU  representatives  work  closely  with  the  career 
field/path  FIPTs  in  their  development  and  modification  of  course  material.  Before 
launching  into  a  more  specific  discussion  about  how  the  SPRDE/SE  FIPT  has  worked 
with  DAU,  an  introduction  to  some  of  DAU’s  instructional  systems  development 
tenninology  will  be  helpful,  as  these  terms  are  used  throughout  this  document. 

1.  Instructional  Terminology 

DAU  representatives  provided  the  Instructional  Systems  Development 
Terminology  in  Table  II-2. 


“Functional  Advisers  ‘Quarterback’  Career  Development,  ARToday,  Vol.  6,  No.  3,  June/July  2001, 
P-1 
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Table  11-2.  Instructional  Systems  Development  Terminology  from  DAU 


Term 

Defined  as... 

Example 

Competency 

A  behavior  or  set  of  behaviors  that  describes  excellent 
performance  in  particular  work  contexts.  Competencies 
are  those  worthy  accomplishments  that  make  the 
employee  valuable  to  the  employer  and  that  make  the 
employer  valuable  to  the  customer. 

Historically  the  Department  of  Defense,  when  referring 
to  training  its  acquisition  workforce  applied  the  term 
“competency-based”  in  attempting  to  determine  training 
and  education  requirements.  Competency-based 
training  and  education  was  viewed  as  job  or  occupation 
specific.  That  is,  an  individual  learner  would  concentrate 
on  mastering  the  “competencies”  associated  with  a 
particular  government  job  series  or  classification  (e.g., 
Contract  Specialist). 

Complete  appropriate 
DoD  forms 

Performance 

Outcomes 

A  statement  of  performance  of  what  the  student  should 
be  able  to  do,  related  to  actual  job  performance 
requirements  along  with  the  conditions  under  which  the 
student  is  to  perform  and  the  criteria  for  acceptable 
performance 

Complete  DoD 

1351-2  within  5  days 
of  returning  from  a 
Temporary  Duty 
assignment 

Learning 

Outcomes 

For  every  education  or  training  program,  and  for  each 
course  and  lesson  within  that  program,  there  is  a  set  of 
intended  learning  outcomes.  These  outcomes  are 
statements  of  what  the  learners  should  be  able  to  do  as 
a  result  of  the  instruction.  The  outcomes  expected  from 
a  course  or  lessons  are  known  as  learning  objectives, 
and  serve  three  most  important  functions.  Learning 
objectives: 

•  Define  the  desired  outcomes  of  learning 

•  Serve  as  a  guide  to  the  selection  of  strategies 
and  methods  of  instruction 

•  Provide  criteria  for  evaluation  of  the  learning 

Terminal 

Learning 

Objectives 

(TLO) 

The  identified  desired  learning  outcomes  after  a 
segment  of  instruction,  generally  a  course  or  a  lesson  or 
module.  TLOs  are  frequently  translated  directly  from 
task  statements  and  clearly  state  the  after-instruction 
performance  the  learner  must  be  able  to  demonstrate. 
They  must  be  observable  and/or  measurable  and 
contain  a  condition,  performance  and  standard. 

Given  a  blank  DoD 
1351-2  form  and  all 
receipts,  accurately 
complete  the  form  so 
that  the  traveler  will 
be  reimbursed  for  all 
entitlement 
expenses. 

Enabling 

Learning 

Objectives 

(ELO) 

Provides  the  means  for  reaching  the  terminal  objectives 
and  generally  consist  of  the  skills,  knowledge,  and  task 
steps  that  must  be  completed  or  satisfied  in  order  to 
master  the  terminal  objective.  Enabling  objectives  are 
derived  from  the  elements,  skill,  and  knowledge 
requirements. 

Describe  the  required 
information  needed  in 
the  travel  voucher. 

Recognize 

reimbursable 

entitlement 

expenses. 
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2.  Performance  Learning  Model 


In  addition  to  the  DAWIA  certification  courses,  DAU  also  offers  other 


educational  resources  to  complement  career  field/path  requirements.  These  resources  are 
available  to  the  acquisition  workforce  anytime  and  anywhere.  DAU’s  Performance 
Learning  Model,  as  it’s  called,  “lays  the  foundation  for  meeting  the  career-long  training 
and  professional  development  needs  of  the  AT&L  workforce.”  Figure  II- 1  shows  a 


picture  of  this  model  from  the  2004  DAU  Course  Catalog. 


c  u 

P 

r  .  ‘ 

Training  Courses 


t'riov/ljUfa 
i>huruli 
■  AKSS  (Dcskbook) 

•  Acquisition  1 

Community 
Connection 

•  DAU  Virtual  Library 


Pur/v/wan'je 

Zujjp'jr. 

•  Consulting 

*  Rapid  Deployment 
^ Training  (RDT) 

cted  Training 


Figure  11-1.  AT&L  Performance  Learning  Model 


Knowledge  Sharing,  Continuous  Learning,  and  Performance  Support  constitute 
the  foundation  of  the  DAU  Performance  Learning  Model.  Performance  support  “is 
tailored  to  the  customer’s  needs  and  includes,  but  is  not  limited  to,  consulting,  coaching, 
mentoring,  and  facilitation.”  “  Included  within  Performance  Support  is  Rapid 
Deployment  Training  (RDT) — an  approach  to  quickly  deliver  training  on  a  particular 
topic.  DAU  can  focus  on  high-value  initiatives  to  quickly  develop  and  deliver  training  to 
the  acquisition  workforce  right  after  the  initiative  is  implemented,  while  updates  are 
being  made  to  certification  courses.  23  Recent  examples  of  RDT  programs  include 
DoD5000,  Corrosion  Prevention  and  JCIDS  3170.01c.24 


21 

22 

23 

24 


DAU  Course  Catalog  2004,  p.  viii 
Ibid. 

Ibid. 

These  programs  can  be  accessed  at  http://view.dau.mil/dauvideo/view/ 
channel.  ihtml?stationID=909876928. 


II-6 


Continuous  Learning  Modules  (CLM)  are  self-paced  modules  and  briefings  that 
DAU  develops  and  makes  available  to  the  Acquisition  workforce  through  the  DAU  Web 
site.  Students  in  DAWIA  career  fields/paths  are  required  to  complete  80  hours  of  these 
Continuous  Learning  courses  every  2  years.  The  courses  are  not  specific  to  a  career 
field/path  but  instead  provide  information  on  particular  topics  of  interest  to  acquisition 
professionals. 

DAU  manages  the  Acquisition  Community  Connection,  Communities  of  Practice, 
and  AT&L  Knowledge  Sharing  System  (Deskbook)  as  a  part  of  the  Knowledge  Sharing 
part  of  the  Performance  Model.  These  Web  sites  provide  an  electronic  forum  for  the 
acquisition  workforce  to  obtain  reference  materials,  lessons  learned,  and  best  practices 
and  to  ask  questions. 

3.  Relationship  to  Defense  Acquisition  Workforce  Improvement  Act  Career 
Field/Path  Functional  Integrated  Product  Teams 

DAU  works  closely  with  the  DAWIA  Career  Field/Path  FIPTs.  As  the  developer 
of  the  DAWIA  education  and  training  materials,  DAU  must  address  the 
recommendations  that  a  FIPT  makes  regarding  those  materials.  The  FIPT  develops  a  set 
of  competencies  that  individuals  in  a  career  field/path  need  to  have  and  then  provides  it  to 
DAU  as  guidance  for  course  development.  The  tenn  “competencies”  is  used  loosely  here: 
the  FIPT  guidance  may  be  in  the  form  of  Performance  Objectives,  Learning  Outcomes  or 
some  other  high-level  course  guidance  (such  as  Duties  and  Tasks)  that  DAU  and  the 
FIPT  feel  is  appropriate.  Using  these  “competencies,”  DAU  then  develops  Terminal 
Learning  Objectives  (TLOs)  and  Enabling  Learning  Objectives  (ELOs)  for  the 
certification  courses.  Course  developers  use  the  TLOs  and  ELOs  as  guidance  in 
developing  the  courses.  DAU  also  works  with  the  FIPTs  in  the  development  of  RDT  and 
CLMs.  After  a  FIPT  recommends  a  new  CLM,  they  generally  provide  a  subject  matter 
expert  to  work  with  DAU  to  develop  the  course. 

C.  SPRDE/SE  REVIEW  PROCESS 

The  most  important  career  field/path  for  systems  engineering  is  the  Systems 
Engineering  career  path  of  the  Systems  Planning,  Research,  Development  and 
Engineering  career  field.  Systems  engineering  material  is  taught  in  other  fields/paths  as 
well,  but  the  SPRDE/SE  career  path  is  the  primary  focus  of  the  review.  Review  of  other 


DAU  Continuous  Learning  Modules  are  available  at  http://clc.dau.mil/kc/no  login/portal. asp. 
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systems  engineering-related  material  in  other  career  fields/paths  and  courses  is  covered  in 
Section  VI.A,  “Next  Steps,”  at  the  end  of  this  document. 

1.  SPRDE/SE  Career  Path 

The  SPRDE  career  field  is  the  largest  of  the  DAWIA  career  fields.  It  includes 
scientists  and  engineers,  typically  working  in  laboratories  and  program  offices, 
performing  such  activities  as  identifying,  establishing,  organizing,  or  implementing 
acquisition  engineering  objectives  and  policies,  or  establishing  specifications.26  In 
addition  to  the  SE  career  path,  the  SPRDE  career  field  has  a  Science  and  Technology 
Management  (STM)  career  path,  which  is  managed  by  a  separate  FA  and  FIPT.  We  did 
not  consider  the  SPRDE/STM  career  path  education  and  training  material  as  a  part  of  this 
review,  although  its  systems  engineering-related  content  will  eventually  be  reviewed 
separately  by  the  SPRDE/SE  FIPT,  as  described  in  Chapter  VI. 

SPRDE/SE  falls  under  the  DAWIA  Technical  Management  functional  area.  The 
Director  of  Systems  Engineering,  Mr.  Mark  Schaeffer,  is  the  Functional  Advisor  for  the 
Technical  Management  functional  area  and  the  three  career  fields/paths  within  it: 
SPRDE/SE;  Production,  Quality,  and  Manufacturing  (PQM);  and  Test  and  Evaluation 
(T&E).  Mr.  Robert  Skalamera,  Deputy  Director  of  Enterprise  Development,  is  the 
Executive  Secretary  of  the  SPRDE/SE  FIPT.  A  listing  of  the  SPRDE  FIPT  membership  is 
shown  in  Table  II-3,  below.  In  addition  to  the  actual  FIPT  members,  the  group  invites 
others,  such  as  the  executive  secretaries  of  other  career  field’s  or  path’s  FIPTs,  to 
participate  in  their  meetings. 


Table  11-3.  SPRDE/SE  FIPT  Members 


SPRDE/SE  FIPT  Members 

Organization/Service 

Brooks  Bartholow 

Army/AMC 

R.  Pillai 

DCMA/OCT 

Lt.  Col  Erica  Robertson 

SAF/AQRE 

John  Shaefer,  Jr. 

Navy/SSP 

Bob  Skalamera  (FIPT  Chair) 

OUSD(AT&L)/DS/SE — Deputy  Director,  Enterprise  Development 

John  Snoderly 

DAU,  SPRDE  Program  Director 

DoDI  5000.28,  “Defense  Acquisition  Workforce”  (1992)  p.  51,  http://www.dau.mil/career/files/5000- 
58i.pdf  (accessed  25  August  2004) 
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IDA’s  participation  in  the  SPRDE/SE  FIPT  began  in  December  2003.  Since  that 
time  IDA  has  hosted  and  facilitated  four  of  the  SPRDE/SE  FIPT  meetings:  1 1  December 
2003,  22  January  2004,  26  February  2004,  and  1  July  2004.  Additional  meetings  were 
held  at  DAU  on  22  July  and  26  August  2004. 

2.  Review  Process 

a.  December  and  January  Functional  Integrated  Product  Team  Meetings 

During  the  SPRDE/SE  FIPT  meetings  on  1 1  December  and  22  January,  one  of  the 
main  topics  of  discussion  was  devising  an  approach  to  review  the  SPRDE/SE  education 
and  training,  as  required  by  the  SPRDE/SE  FIPT  charter.  The  group  decided  that  they 
should  review  the  PCDs;  education,  training  and  experience  (ET&E)  requirements; 
Performance  Objectives  (POs);  and  Terminal  Learning  Objectives  (TLOs)  for  all  of  the 
DAWIA  certification  courses — even  those  outside  the  SPRDE/SE  career  path — that 
include  systems  engineering-related  material.  The  review  would  be  conducted  by 
certification  level,  starting  with  the  SPRDE/SE  career  path. 

One  of  the  DAU  representatives  at  the  meeting  suggested  that  the  first  step  should 
be  to  review  the  existing  “competencies”  for  the  courses.  These  competencies  are  the 
high-level  guidance  from  which  DAU  develops  more  detailed  TLOs  and  ELOs.  IDA 
encouraged  the  FIPT  to  use  a  systems  engineering  approach  and  emphasized  the  need  to 
examine  the  existing  competencies  against  some  logical  criteria.  IDA  recommended  first 
looking  at  what  today’s  SPRDE/SE  workforce  needs  to  do — given  the  current  DoD 
environment,  policy  and  initiatives.  The  next  step  would  be  to  perform  a  gap  analysis 
between  what  the  SPRDE  workforce  needs  to  do  and  the  existing  SPRDE/SE 
competencies.  If  the  systems  engineering  workforce  is  considered  the  “system”  here,  IDA 
proposed  defining  the  functions  that  the  workforce  system  must  perform — a  functional 
architecture  of  sorts.  The  FIPT  agreed  to  this  approach,  and  a  DAU  representative 
explained  that  another  career  field,  BCEFM,  followed  a  similar  approach,  developing  the 
competencies  in  the  form  of  “duties  and  tasks”  that  the  SPRDE/SE  workforce  performs. 
DAU  provided  the  BCEFM  “Duties  and  Tasks”  list  to  IDA  as  an  example  to  follow.  Note 
that  the  term  “duties  and  tasks”  does  not  exist  in  the  ISD  tenninology  table  earlier  in  this 
chapter.  DAU  does  not  appear  to  have  been  consistent  in  its  use  of  ISD  tenninology,  so 
we  were  not  overly  concerned  about  these  tenns.  Regardless  of  the  terms  used,  the  point 
was  to  provide  DAU  with  requirements  for  what  a  student  should  learn/  be  able  to  do  at  a 
given  course  level. 
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Ideally,  IDA  would  have  conducted  a  requirements  analysis  workshop  to 
determine  what  the  SPRDE/SE  workforce  needs  to  know  and  do.  Early  in  our  work,  we 
consulted  with  another  team  in  the  Cost  Analysis  and  Research  Division  of  IDA  that 
performed  an  assessment  of  the  Program  Manager  (PM)  courses  for  DAU.  That  IDA 
team  interviewed  the  students  who  took  the  PM  course,  as  well  as  some  of  their 
supervisors,  90-120  days  after  graduation,  asking  them  about  the  relevance  and 
effectiveness  of  the  PM  courses  being  assessed.  Unfortunately,  we  were  unable  to  pursue 
such  activities  for  the  SPRDE/SE  review  due  to  time  constraints.  Instead,  we  relied  on  the 
SPRDE/SE  FIPT  members  to  ensure  that  the  functions/duties  and  tasks  were  accurate. 

b.  Proposed  IDA  Overall  Approach 

During  the  26  February  FIPT,  IDA  proposed  an  approach  to  review  the  SPRDE 
education  and  training  requirements,  guidance,  and  materials  using  a  combined  top-down 
and  bottom-up  approach.  Figure  II-2,  below,  shows  a  depiction  of  that  approach.  It  was 
approved  by  the  SPRDE  FIPT. 


Top-Down 


Education  and  Training 

Changes  & 
Recommendations 


Bottom-Up 


Continuous 
Learning  Module s. 


Rapid  Deployment 
Training 


PCD 


Certification  Level 
Requirements 


SPRDE/SE 
Duties  &  Tasks 

ourse  Organization 
&  Content 


Ch  IV 


Ch  IV 


Ch  III 


Ch  III 


Ch  V 


Ch  IV 


Figure  11-2.  Overview  of  Proposed  Approach  to  SPRDE/SE  Course  Review 
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The  idea  behind  the  review  was  to  merge  a  top-down  and  a  bottom-up  approach  to 
support  a  gap  analysis  and  generate  recommendations  to  the  FIPT  regarding  changes  to 
the  education  and  training  elements  of  the  SPRDE/SE  career  path.  The  top-down  review, 
or  review  of  “The  SE  Problem,”  would  take  into  consideration  DoD  and  systems 
engineering  policy  guidance,  such  as  the  “hooks,”  the  systems  engineering  chapter  of  the 
Acquisition  Guidebook,  the  Defense  Acquisition  Framework  wallchart,  and  non-DoD 
best  practice  resources.  The  top-down  review  is  described  later,  in  Chapter  V.  The 
bottom-up  review,  on  the  other  hand,  would  be  conducted  through  a  review  of  current 
SPRDE/SE  certification  course  content  (discussed  in  Chapter  IV)  and  review  of  past  and 
existing  competencies/TLOs/ELOs  for  those  certification  courses  (discussed  in  Chapter 
V).  Looking  at  what  exists  in  the  courses  compared  with  what  should  exist  is  the  gap 
analysis.  The  ovals  on  the  right  side  of  the  diagram  are  the  SPRDE/SE  education  and 
training  elements  to  be  reviewed  and  revised  through  the  process.  Each  of  these  ovals 
will  be  addressed  in  more  detail  in  the  chapters  indicated  in  the  figure. 

A  distinction  should  be  drawn  at  this  point  regarding  the  implementation  of  this 
approach.  IDA  was  specifically  responsible  for  the  development  of  the  SPRDE/SE 
competencies  in  the  fonn  of  “Duties  and  Tasks”  that  SPRDE/SE  personnel  perform. 
Modification  of  other  elements,  such  as  the  Continuous  Learning  Modules,  Rapid 
Deployment  Training,  and  PCD  and  certification  level  requirements  was  done  by  the 
FIPT.  The  approach  documented  here,  which  IDA  proposed  and  followed  in  developing 
the  SPRDE/SE  duties  and  tasks,  was  not  necessarily  followed  by  others  in  their  parts  of 
the  review. 
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III.  SPRDE/SE  CAREER  PATH  GUIDANCE  DOCUMENTS 


Two  of  the  primary  SPRDE  career  field  guidance  documents,  the  Position 
Category  Description  (PCD)  and  the  Education,  Training  and  Experience  (ET&E) 
certification  requirements,  are  described  in  this  chapter.  The  SPRDE  FIPT  has  proposed 
revisions  to  both  the  PCD  and  the  certification  requirements.  These  revisions  were 
submitted  to  the  Career  Management  Overarching  Integrated  Product  Team  (CMOAIPT) 
for  approval.  27 

A.  POSITION  CATEGORY  DESCRIPTION 

A  PCD  describes  the  acquisition  duties  that  an  individual  working  within  a 
DAWIA  career  field/path  is  likely  to  perfonn.  The  descriptions  are  generally  defined  by 
these  duties  rather  than  specific  job  title  or  occupational  series. 

The  SPRDE/SE  PCD  was  as  follows: 

Plan,  organize,  monitor,  manage,  oversee,  and/or  perform  research  and/or 
engineering  activities  relating  to  the  design,  development,  fabrication, 
installation,  modification,  or  analysis  of  systems  or  systems  components. 

Duties  may  require  identification,  establishment,  organization,  or 
implementation  of  acquisition  engineering  objectives  and  policies,  or 
establishing  of  specifications.  Scientists  and  engineers  directly  supporting 
acquisition  programs,  projects,  or  activities  (including  medical)  usually 
accomplish  these  duties. 

The  SPRDE/SE  FIPT  revised  the  PCD  to  stress  integration,  emphasize  the  entire 
life  cycle,  and  reinforce  the  application  of  systems  engineering  to  acquisition.  The 
proposed  revision  reads  as  follows — 

Plan,  organize,  monitor,  manage,  oversee  and/or  perform  research  and/or 
engineering  relating  to  the  analysis,  design,  development,  integration, 
verification  and  validation,  modification,  and  sustainment  and  support  of 
systems  or  system  components  across  the  entire  system  life  cycle.  Duties 
may  require  identification,  integration,  or  implementation  of  acquisition 
engineering  (systems  engineering  as  applied  to  acquisition)  objectives, 
policies,  guidelines,  and  processes  spanning  both  internal  (acquirer)  and 
external  (supplier)  domains.  Scientists  and  engineers  directly  supporting 


From  AnnMarie  Choepel’s  minutes  of  the  26  February  SPRDE  FIPT  meeting 
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acquisition  programs,  projects,  or  activities  usually  accomplish  these 
duties. 


B.  EDUCATION,  TRAINING  AND  EXPERIENCE  CERTIFICATION 
REQUIREMENTS 

Certification  requirements  for  the  SPRDE/SE  career  path  are  shown  in  Table 
III- 1 .  Three  levels  of  certification  are  available,  with  education,  experience,  and  training 
requirements  for  each  as  shown.  The  courses  in  the  Training  column  are  discussed  further 
in  Chapter  IV. 


Table  111-1.  SPRDE/SE  Certification  Requirements 


Education 

Training 

Experience 

Level  1 

Baccalaureate  degree  in  engineering, 
physics,  chemistry,  mathematics,  or  a 
related  field;  or  at  least  10  years  of 
acquisition  experience  in  Systems  Planning, 
Research,  Development  and  Engineering 
(as  of  1  October  1991) 

ACQ  101  -  Fundamentals 
of  Systems  Acquisition 
Management 

1  year  of 
acquisition 
experience 
in  science  or 
engineering 

Level 

II 

Baccalaureate  degree  in  engineering, 
physics,  chemistry,  mathematics,  or  a 
related  field;  or  at  least  10  years  of 
acquisition  experience  in  Systems  Planning, 
Research,  Development  and  Engineering 
(as  of  1  October  1991) 

(Desired)  Master’s  degree  in  engineering, 
physics,  chemistry,  mathematics,  operations 
research,  management,  or  a  related  field 
(Desired)  9  semester  hours  from  among 
accounting,  business  finance,  law, 
economics,  industrial  management, 
quantitative  methods,  or  organization  and 
management.  (DANTES  or  CLEP  exams 
may  be  substituted) 

ACQ  201  Intermediate 
Systems  Acquisition 

SYS  201  Intermediate 
Systems  Planning, 
Research,  Development 
and  Engineering 
(Desired)  A  DAU  Level 

200  or  Level  100  course 
mandatory  for  Acquisition 
Logistics;  Program 
Management; 

Production,  Quality  and 
Manufacturing; 

Information  Technology; 
or  Test  and  Evaluation 

2  years  of 
acquisition 
experience 
in  science  or 
engineering 
(Desired)  An 
additional  2 
years  of 
acquisition 
experience 
in  science  or 
engineering 

Level 

III 

Baccalaureate  degree  in  engineering, 
physics,  chemistry,  mathematics,  or  a 
related  field;  or  at  least  10  years  of 
acquisition  experience  in  Systems  Planning, 
Research,  Development  and  Engineering 
(as  of  1  October  1991) 

(Desired)  Advanced  degree  in  engineering, 
physics,  chemistry,  mathematics,  operations 
research,  management  or  a  related  field 

(Desired)  12  semester  hours  from  among 
accounting,  business  finance,  law, 
economics,  industrial  management, 
quantitative  methods,  or  organization  and 
management.  (DANTES  or  CLEP  exams 
may  be  substituted) 

SYS  301  Advanced 
Systems  Planning, 
Research,  Development 
and  Engineering 
(Desired)  Any  mandatory 
DAL)  Level  200  or  Level 
300  course  in  Acquisition 
Logistics;  Program 
Management; 
Manufacturing, 

Production,  and  Quality 
Assurance;  Information 
Technology;  or  Test  and 
Evaluation 

4  years  of 
acquisition 
experience 
in  science  or 
engineering 

(Desired) 

4  additional 
years  of 
experience 
in  acquisition 
positions  of 
increasing 
responsibility 
and 

complexity 
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The  SRPDE  FIPT  made  one  small  revision  to  these  requirements  by  adding 
“biology”  as  an  acceptable  degree  for  the  SPRDE  education  requirements.  The  education 
requirement  now  reads: 

Baccalaureate  degree  in  engineering,  physics,  chemistry,  biology, 
mathematics,  or  a  related  field;  or  at  least  10  years  of  acquisition 
experience  in  Systems  Planning,  Research,  Development  and  Engineering 
(as  of  1  October  1991) 

Another  change  was  made  to  the  training  requirements — the  addition  of  a  SYS 
101  course  to  the  level  1  certification  requirements.  Chapters  IV  and  V  talk  more  about 
this  decision.  After  the  training  piece  is  more  under  control,  the  FIPT  may  wish  to  revisit 
the  education  and  experience  requirements  again. 
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IV.  SPRDE/SE  TRAINING  RESOURCES 


SPRDE/SE  training  resources  include  both  the  certification  courses  and  other 
systems  engineering-related  materials  developed  by  DAU,  such  as  Continuous  Learning 
Modules  and  Rapid  Deployment  Training.  This  chapter  describes  the  existing  and 
proposed  training  resources  for  the  SPRDE/SE  career  path  as  well  as  content  reviews 
IDA  conducted  on  Systems  Engineering  (SYS)  201  A&B  and  SYS  301. 

A.  SPRDE/SE  CERTIFICATION  COURSES 

DAU  develops  DAWIA  certification  courses  for  the  various  career  fields  and 
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paths  using  the  following  certification  guidelines: 

•  Level  1  courses  are  “designed  to  provide  fundamental  knowledge  and 
establish  primary  qualification  and  expertise  in  the  individual’s  career  field, 
job  series  or  functional  area.” 

•  Level  2  courses  focus  on  “functional  specialization”  and  are  designed  to 
“enhance  the  employee’s  capabilities  in  a  primary  specialty  or  functional 
area.” 

•  Level  3  courses  focus  on  “managing  the  acquisition  process  and  learning  the 
latest  methods  being  implemented  in  the  career  field  or  functional  area.” 

DAU’s  current  (2004)  course  offerings  in  the  SPRDE/SE  career  path  are  shown  in 
Table  III- 1  in  the  previous  chapter.  They  include  ACQ  101  for  Level  I  certification,  ACQ 
201A/ACQ  20 IB  and  SYS  201A/SYS201B  for  Level  II  certification,  and  SYS  301  for 
Level  III  certification.  ACQ  201  A  and  SYS  201  A  are  on-line,  distance  learning 
components  of  their  ACQ  201  B  and  SYS  201  B  resident/on-site  classroom-taught 
counterparts.  The  ACQ  courses  are  general  acquisition  courses  that  form  the  basis  for 
most  acquisition  career  fields/paths.  Unlike  the  SYS  courses,  which  are  fundamental  to 
the  SPRDE/SE  career  path,  the  ACQ  courses  do  not  relate  to  a  single  career  field/path. 

At  the  time  of  this  review,  SPRDE/SE  level  I  certification  did  not  include  a  SYS 
course.  However,  the  FIPT  was  interested  in  developing  one,  and  DAU  was  already 
considering  its  development.  At  the  direction  of  Bob  Skalamera,  Deputy  Director  of 
Enterprise  Development,  IDA  presented  the  SPRDE/SE  Career  Path  and  Certification 
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Level  model  shown  in  Figure  IV- 1  as  a  part  of  its  review  approach  presentation.  In  the 
proposed  model,  level  I  courses  would  teach  the  theoretical  basis  for  systems 
engineering,  covering  systems  engineering  revitalization  initiatives  and  knowing  how  to 
participate  in  systems  engineering  activities.  The  level  II  courses  would  teach  what  to  do 
in  each  life  cycle  phase,  including  sustainment  engineering.  The  level  III  course  would 
cover  systems  engineering  management  and  leadership  capability,  including  knowing 
how  to  lead  a  review  and  perfonn  the  duties  of  chief  engineer.  The  chart  below,  presented 
at  the 

26  February  FIPT,  shows  different  options  to  achieve  the  certification  level  goals.  The 
options  were  based  on  whether  or  not  the  SPRDE/SE  FIPT  opted  to  develop  a  new  SYS 
101  course.  The  options  were  1)  stay  with  the  status  quo,  2)  develop  a  new  SYS  101 
course  for  level  I,  and  3)  split  the  SYS  201  A  and  B  courses  up  between  levels  I  and  II. 

Recommended  SPRDE/SE  Career  Path  & 
Certification  Approach 

□  Level  I:  SE  Process 

■  Theoretical  basis 

■  Course  Options: 

□  ACQ101  [current] 

□  ACQ101  and  SYS101  OR 

□  ACQ  101  and  SYS201A? 

□  Level  II:  SE  by  Life  Cycle 

■  What  to  do  in  each  life  cycle  phase  (including 
sustainment  engineering) 

■  Covers  some  of  the  revitalization  "hooks" 

■  Course  Options: 

□  ACQ201  and  SYS201A&B  [current]  OR 

□  ACQ201  and  SYS201B? 

□  Level  III:  Lead  SE 

■  Management  and  leadership  capability 

■  Covers  some  of  the  revitalization  "hooks" 

■  Course:  SYS301  [current] 


Figure  IV-1.  Recommended  SPRDE/SE  Career  Path  and  Certification  Approach 


The  SPRDE/SE  FIPT  did  choose  to  develop  the  SYS  101  course;  the  chosen 
options  on  the  graphic  are  marked  with  arrows.  Much  of  the  new  course  material 
resulting  from  changes  to  systems  engineering  policy  and  systems  engineering 
revitalization  “hooks”  would  be  covered  in  the  level  II  SYS  course.  For  example,  the 
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Defense  Acquisition  Framework  (wall  chart)  resulted  in  a  significant  amount  of  new 
guidance  about  how  systems  engineering  should  be  conducted  throughout  the  acquisition 
phases.  More  basic,  theoretical  systems  engineering  process  material  that  had  originally 
been  covered  in  the  level  II  SYS  201 A  online  course  could  now  be  moved  into  the  level  I 
course,  SYS  101.  However,  because  of  a  lag  between  enrollments  of  each  of  these 
courses,  it  is  recognized  that  a  clear  division  of  subject  matter  across  the  three  levels  is 
not  possible  and  there  will  necessarily  be  some  overlap. 

This  next  section  includes  the  course  descriptions,  DAU  course  catalog  narratives, 
and  objectives  for  the  courses  required  for  SPRDE/SE  certification. 

1.  Level  1 
a.  ACQ  101 

The  DAU  description  for  ACQ  101 : 

•  For  military  officers,  0-1  through  0-3,  and  DoD  civilians,  GS-5  through 
GS-9.  However,  the  course  is  open  to  all  ranks  and  grades 

•  Required  for  Level  I  certification  in  SPRDE/SE  career  path 

•  No  prerequisites 

•  Distance  Learning 

The  2004  DAU  course  catalog  narrative  for  the  ACQ  101  course  includes  the 
following: 

This  course  provides  a  broad  overview  of  the  DoD  systems  acquisition 
process,  covering  all  phases  of  acquisition.  It  introduces  the  requirements 
generation  and  resource  allocation  processes,  the  DoD  5000  Series 
documents  governing  the  defense  acquisition  process,  and  current  issues  in 
system  acquisition  management.  Designed  for  individuals  who  have  little 
or  no  experience  in  DoD  acquisition  management,  ACQ  101  has  proven 
very  useful  to  personnel  in  headquarters,  program  management,  and 
functional  or  support  offices. 

In  addition,  students  who  successfully  completed  the  existing  ACQ  101  course 
would  be  able  to  recognize — 

•  The  fundamental  precepts  and  bases  of  defense  systems  acquisition 
management 

•  The  diverse,  interrelated,  and  changing  nature  in  the  different  disciplines  of 
defense  systems  acquisition  management 

•  The  regulations  and  governing  structures  of  defense  systems  acquisition 
management 
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b.  Proposed  SYS  101 

As  discussed  in  the  previous  chapter,  the  addition  of  a  SYS  101  course  was 
desired  among  the  DoD  systems  engineering  community.  DAU  felt  that  this  addition 
would  be  fairly  straightforward  because  a  lot  of  the  material  currently  taught  in  the  SYS 
201A  on-line  course  could  be  incorporated  into  a  SYS  101  course. 

2.  Level  2 

a.  ACQ201A 

The  DAU  description  for  ACQ  201  A: 

•  For  military  officers,  0-3  and  above;  civilians,  GS-9  and  above;  and  industry 
equivalents  who  are  Level  I  certified  in  acquisition.  Students  should  have  2  to 
4  years  of  acquisition  and/or  logistics  experience 

•  Required  for  Level  II  certification  in  SPRDE/SE  career  path 

•  Prerequisite  ACQ  101 

•  Distance  Learning 

The  2004  DAU  course  catalog  narrative  for  the  ACQ  201 A  course: 

Intermediate  Systems  Acquisition,  Part  A,  uses  computer-based  training  to 
prepare  mid-level  acquisition  professionals  to  work  in  integrated  product 
teams  by  understanding  systems  acquisition  principles  and  processes.  Both 
ACQ201A  and  ACQ201B  are  required  for  DAWIA  certification. 

In  addition,  students  who  successfully  completed  the  existing  ACQ  201 A  course 
would  be  able  to — 

•  Enhance  their  knowledge  of  the  business,  technical,  and  managerial  aspects  of 
acquisition; 

•  Understand  and  appreciate  the  critical  role  that  each  functional  discipline 
plays  in  the  acquisition  process;  and 

•  Using  computer-based  training,  theoretically  participate  in  simulated 
integrated  product  teams  to  develop  plans  and  resolve  problems 

b.  ACQ201B 

The  DAU  description  for  ACQ  20 IB: 

•  For  military  officers,  0-3  and  above;  civilians,  GS-9  and  above;  and  industry 
equivalents  who  are  Level  1  certified  in  acquisition.  Students  should  have  2  to 
4  years  of  acquisition  and/or  logistics  experience. 

•  Required  for  Level  II  certification  in  SPRDE/SE 

•  Prerequisite  ACQ  201 A 

•  Resident/on-site,  5  class  days 
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The  2004  DAU  course  catalog  narrative  for  the  ACQ  20 IB  course  includes  the 
following: 

Intermediate  Systems  Acquisition,  Part  B,  prepares  mid-level  acquisition 
professionals  to  work  effectively  in  integrated  product  teams  by 
understanding  systems  acquisition  principles  and  processes.  Both 
ACQ201A  and  ACQ201B  are  required  for  DAWIA  certification 

In  addition,  students  who  successfully  completed  the  existing  ACQ  20 IB  course 
would  be  able  to — 

•  Enhance  and  apply  their  knowledge  of  the  business,  technical,  and  managerial 
aspects  of  acquisition; 

•  Understand  and  appreciate  the  critical  role  that  each  functional  discipline 
plays  in  the  acquisition  process;  and 

•  Effectively  participate  in  integrated  product  teams  and  apply  knowledge 
gained  in  ACQ201A  to  develop  plans  and  resolve  problems 

c.  SYS201A 

The  DAU  description  for  SYS  201  A: 

•  Introductory,  theoretical 

•  Required  for  Level  II  certification  in  SPRDE/SE  career  path 

•  Prerequisite  ACQ  20 IB 

•  Distance  learning 

The  2004  DAU  course  catalog  narrative  for  the  SYS  201 A  course  includes  the 
following: 

This  journeyman-level  course  exposes  students  to  systems  engineering  and 
associated  topics.  Course  content  includes  the  systems  engineering 
process;  systems  engineering  planning;  technology  insertion;  risk 
management;  trade  studies;  configuration  management;  cost  containment; 
technical  reviews;  and  Environmental,  Safety,  and  Occupational  Health 
(ESOH). 

In  addition,  students  who  successfully  completed  the  existing  SYS  201 A  course 
would  be  able  to — 

•  Understand  the  systems  engineering  process 

•  Know  the  associated  systems  engineering  technical  activities 

•  Evaluate  a  Hazardous  Material  Management  Plan  and  identify  ESOH  issues 
that  need  further  clarification,  and 

•  Develop  and  defend  a  technical  review  checklist 
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d.  SYS201B 


The  DAU  description  for  SYS  20 IB  is — 

•  Practical 

•  Required  for  Level  II  certification  in  SPRDE/SE  career  path 

•  Prerequisite  SYS  201 A 

•  Five  class  days,  resident  on-site 

The  2004  DAU  course  catalog  narrative  for  the  SYS  20 IB  course  includes  the 
following: 

This  journeyman-level  course  requires  students  to  apply  the  Systems 
Planning,  Research,  Development  and  Engineering  processes  and 
techniques  learned  in  SYS  201  A.  Students  will  work  in  integrated  product 
teams  to  apply  the  systems  engineering  process  and  its  associated 
technical  activities. 

In  addition,  students  who  successfully  completed  the  SYS  20 IB  course  would  be 
able  to — 

•  Conduct  a  requirements  analysis  for  a  given  need 

•  Prepare  Functional  Analysis  and  Allocation  and  Synthesis  tools  for  a  given 
scenario 

•  Apply  the  acquisition  risk  management  process 

•  Propose  trade  study  methodologies 

•  Develop  technical  performance  measures 


3.  Level  3 
a.  SYS  301 

The  DAU  description  for  SYS  301: 

•  For  DoD  civilians,  GS-13  and  above,  and  military  officers,  0-3  to  0-6,  who 
are  Level  II  certified  in  the  SPRDE/SE  career  path.  Equivalent  industry 
acquisition  managers  are  also  eligible 

•  Prerequisite  SYS  201 

•  10  class  days,  resident  on-site 

The  2004  DAU  course  catalog  narrative  for  the  SYS  301  course  is  as  follows: 

Designed  for  senior  DoD  acquisition  personnel,  this  course  emphasizes  an 
understanding  of  science,  technology,  and  the  systems  engineering 
processes  throughout  a  system’s  life  cycle  by  using  relevant  case  studies 
and  exercises  involving  all  acquisition  phases  and  milestones.  Participants 
employ  the  proven  principles  and  tools  of  systems  engineering 
requirements  analysis,  risk  management,  technical  performance  measures, 
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tradeoff  analyses,  configuration  and  data  management,  and  technical 
reviews.  Advanced  tools,  such  as  integrated  product  teams,  modeling  and 
simulation,  and  open  systems  architectures,  further  facilitate  managing  and 
developing  the  system. 

In  addition,  students  who  successfully  completed  the  SYS  301  course  would  be 
able  to — 

•  Analyze  and  solve  senior-level  technical  problems 

•  Forecast  cost,  schedule,  performance,  and  risk  issues  across  the  acquisition 
life  cycle 

•  Integrate  program  office  activities 

•  Manage  technical  obsolescence,  advanced  technology  tools,  and  acquisition 
reform  implementation 

4.  IDA  Course  Content  Reviews 

As  a  part  of  the  bottom-up  review  of  current  SPRDE/SE  education  and  training 
material,  IDA  reviewed  the  content  of  the  SYS  201A  and  B  and  SYS  301  courses.  We 
did  not  have  specific  criteria  against  which  to  review  the  courses,  but  conducting  these 
reviews  was  a  way  to  familiarize  ourselves  with  the  course  material,  ask  questions,  and 
make  general  suggestions,  including  highlighting  those  areas  where  the  material  needed 
to  be  updated.  In  addition  to  the  general  reviews,  IDA  also  reviewed  all  three  of  these 
existing  courses  against  the  systems  engineering  revitalization  “hooks.”  Both  of  these 
reviews  were  provided  to  the  Enterprise  Development  office  and  to  DAU.  The  reviews 
can  be  found  in  Appendixes  C  (SYS  201 A  and  B),  D  (SYS  301),  and  E  (SYS  201 A  and  B 
and  301  reviewed  by  hook). 

B.  ADDITIONAL  SPRDE/SE  RESOURCES 

Even  though  the  additional  systems  engineering  resources  that  DAU  develops, 
such  as  CLMs,  are  not  tied  to  a  specific  career  field/path,  the  DAWIA  Functional 
Advisors  and  FIPTs  are  involved  in  their  development.  Chapter  II  describes  some  of  the 
resources  DAU  develops  to  complement  DAWIA  certification.  This  chapter  documents 
recommendations  that  the  SPRDE/SE  FIPT  made  regarding  these  materials. 


Although  continuous  learning  modules  are  not  currently  required  for  certification  in  any  of  the  career 
field/paths,  specific  CLMs  may  be  recommended  for  a  particular  career  path/field.  The  DAU 
Continuous  Learning  Center  website  has  a  capability  that  allows  the  user  to  search  for  CLMs 
recommended  for  a  particular  career  field/path. 
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1.  Continuous  Learning  Modules 

Each  of  the  Functional  Areas  is  allotted  approximately  three  CLMs  each  year. 
Technical  Management’s  three  modules  for  FY04  were  ISO  9000,  Reliability  and 
Maintainability,  and  Joint  and  Multi-Test  and  Evaluation.  After  the  SPRDE  FIPT  was 
allotted  two  additional  CLMs  for  FY04,  the  group  proposed  several  additional  CLM 
topics,  including  Technical  Reviews,  System  Safety  Hazard  Analysis,  Cost 
Considerations  in  Systems  Engineering,  Systems  Engineering  for  Program  Success, 
Systems  Engineering  Planning,  and  Contracting  for  Systems  Engineering.  The 
SPRDE/SE  FIPT  prioritized  the  proposals  at  the  26  February  meeting,  after  which 
Technical  Reviews  and  System  Safety  Hazard  Analyses  were  selected  and  scheduled  for 
development  in  2004.  As  the  FY04  CLMs  are  being  developed  and  finalized,  proposals 
for  the  FY  05  SPRDE/SE  allotment  of  CLMs  are  already  being  solicited.  There  are 
already  several  systems  engineering-related  CLMs,  such  as  “Joint  Capabilities  Integration 
and  Development  System  (JCIDS),”  “Lean — Six  Sigma,  Evolutionary  Acquisition  (EA): 
the  What  &  Why  of  EA,”  “Reliability  and  Maintainability,”  “Value  Engineering,”  and 
“Introduction  to  Interoperability”  to  name  a  few. 

2.  Rapid  Deployment  Training 

The  SPRDE  FIPT  proposed  using  Rapid  Deployment  Training  (RDT)  to 
communicate  to  the  acquisition  workforce  the  new  systems  engineering  policy  and  SEP 
guidance  from  Mr.  Wynne’s  and  Dr.  Lamartin’s  memorandums.  As  the  name  indicates, 
RDT  courses  are  intended  to  reach  the  acquisition  community  quickly.  The  DAU  team 
indicated  that  roughly  45  days  are  needed  to  develop  such  an  RDT  course.  Current  RDT 
modules  are  available  at  http://www.dau.mil/performance  support/RDT.asp. 

3.  Systems  Engineering  Community  of  Practice  (SECoP) 

DAU  manages  a  Systems  Engineering  Community  of  Practice,  available  at 
http://acc.dau.mil/simplify/ev  en.php.  The  SECoP  is  part  of  a  larger  community  of 
practice  that  includes  other  acquisition-related  subject  areas  such  as  Program 
Management  and  Contract  Management.  This  website  provides  a  general  resource  for 
systems  engineering-related  information.  At  one  of  the  early  FIPT  meetings,  DAU 
offered  space  on  the  CoP  as  workspace  for  the  SPRDE/SE  FIPT.  The  FIPT  accepted  the 
workspace  and  two  of  the  SPRDE  FIPT  members  were  granted  “editor”  privileges  to  the 
SECoP. 
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4.  Systems  Engineering  Resources  and  Top-down  Approach 

The  foregoing  additional  SPRDE/SE  resources  are  primarily  driven  by  the  top- 
down  part  of  the  review  process,  as  shown  in  Figure  IV-2.  The  top-down  approach  is  a 
review  of  “The  SE  Problem”  that  takes  into  account  the  current  acquisition  environment 
(e.g.,  changes  to  DoD  5000),  systems  engineering-related  policy  changes,  and  special 
initiatives,  such  as  safety.  The  CLMs  and  RDTs  convey  these  special  initiative  topics  and 
changes  to  policy  using  the  CLMs  or  RDT — whichever  is  more  appropriate  for  the 
situation. 


Education  and  Training 

Changes  & 

Top-Down  Recommendations 


Figure  IV-2.  SE  Resources  and  Top-down  Approach 
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V.  SPRDE/SE  COMPETENCIES 


This  chapter  describes  how  IDA  developed  the  SPRDE/SE  competencies  in  the 
form  of  “duties  and  tasks”  that  the  SPRDE/SE  personnel  must  perform. 

A.  IDA’S  PROCESS  FOR  DEVELOPING  SPRDE/SE  DUTIES  AND  TASKS 

As  described  in  Chapter  II,  IDA  proposed  a  process  for  reviewing  SPRDE/SE 
education  and  training.  As  a  part  of  this  overall  approach,  IDA  suggested  a  process  for 
developing  the  SPRDE/SE  Duties  and  Tasks.  This  included  the  following  steps: 

•  Top-down  Approach 

—  Compiling  and  mapping  lists  of  processes  and  activities  from  systems 
engineering  standards,  handbooks,  and  guides 

—  Compiling  recognized  best  practices  or  deficiencies  in  systems 
engineering  implementation  from  various  sources 

—  Reviewing  systems  engineering  policy  and  guidance 

•  Bottom-up  Approach 

—  Reviewing  existing  SPRDE  course  material 

—  Reviewing  existing  and  past  SPRDE  competencies,  TLOs/ELOs,  POs, 
etc. 

•  Combining  these  efforts  in  order  to  compile  a  list  of  SPRDE/SE  Duties  and 
Tasks 

•  Organizing  SPRDE/SE  duties  and  tasks  into  levels  described  in  Chapter  IV 

—  Level  1 :  SE  Processes 

—  Level  2:  SE  by  Life  Cycle  Phase 

—  Level  3:  Lead  SE 

Once  the  duties  and  tasks  were  developed,  we  envisioned  that  the  SPRDE/SE 
FIPT  would  review  them  to  ensure  that  they  accurately  reflected  the  needs  of  the 
SPRDE/SE  workforce.  Following  the  FIPT  review,  these  duties  and  tasks  would  be 
provided  to  DAU  as  guidance  from  the  FIPT  for  course  content  development.  The  first 
step  in  this  process  was  to  compile  a  list  of  sources  for  the  review.  These  sources  were 
presented  to  the  FIPT  at  the  26  February  meeting  along  with  the  overall  review  approach. 
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1.  Resources  for  the  Top-Down  Review 


In  addition  to  the  DoD  and  systems  engineering  policy,  including  the  “hooks,” 
IDA  collected  a  list  of  other  systems  engineering  resources  to  consult  for  the  top-down 
review.  These  resources  are  listed  here  in  two  general  categories:  those  that  covered 
systems  engineering  processes  and  activities  and  those  that  document  best  practices  and 
deficiencies. 

On  SE  Processes  and  Activities 

•  International  Council  on  Systems  Engineering’s  (INCOSE)  Systems 
Engineering  Book  of  Knowledge  (SEBoK)  Competencies  and  Handbook 

•  Capability  Maturity  Model  Integration  (CMMI)-Systems  Engineering/ 
Software  Engineering/Integrated  Product  and  Process  Development 
(SE/SW/IPPD) 

•  Marine  Corps  System  Command  (MCSC)  Engineering  Certification 
Guidelines 

•  ISO/IEC  15288,  Systems  Engineering — System  Life  Cycle  Processes 

•  EIA  632,  Processes  for  Engineering  a  System 

•  IEEE  1220,  Application  and  Management  of  the  Systems  Engineering  Process 

•  Boeing  Commercial  Sources  data  base 

•  National  Security  Agency’s  (NSA)  Systems  Engineering  Training  program 
requirements 

•  IBM  certification  program  requirements 

•  Other  systems  engineering  certification  program  requirements  (IDA  research) 

•  Systems  engineering  college  textbooks 

On  Best  Practices  and  Deficiencies 

•  Logistic  Management  Institute  (LMI)  Future  Acquisition  and  Technology 
Workforce  Report 

•  Tri-Service  Assessment  Initiative  analyses 

•  Government  Accounting  Office  (GAO)  Best  Practices  Reports 

—  Strong  Management,  Processes,  and  Metrics  Needed  to  Improve 
Software  Acquisition 

—  Defense  Acquisition:  Employing  Best  Practices  Can  Shape  Better 
Weapon  Systems  Decisions 

—  DoD  Teaming  Practices  not  Achieving  Potential  Results 

—  A  Constructive  Test  Approach  is  Key  to  Better  Weapon  System 
Outcomes 

—  Software  and  Systems  Process  Improvement  programs  Vary  in  Use  of 
Best  Practices 
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•  Defense  Science  Board  (DSB)  Reports 
—  Task  force  on  Defense  Software 

■  Dayton  Aerospace  Quick  Study  Report  on  the  Health  of  Systems  Engineering 
Process  Application 

IDA  presented  the  process  and  activities  resources  and  the  best  practices  and 
deficiencies  in  systems  engineering  resources  to  the  FIPT  and  requested  that  the  FIPT 
provide  other  additional  sources  that  would  be  useful  in  the  review.  IDA  compiled 
material  from  the  process-  and  activity-related  resources  above  into  a  large  table  of 
“Systems  Engineering  Topics.”  This  table  is  broken  up  into  segments  and  attached  in 
Appendix  F. 

The  FIPT  recommended  one  additional  source  to  the  best  practices  and 
deficiencies  resources— the  Space  Broad  Area  Review  (Space  BAR).  IDA  compiled  the 
systems  engineering  recognized  best  practices  and  deficiencies  from  these  resources  into 
a  table  titled  “Identified  Systems  Engineering  Best  Practices  or  Deficiencies,”  which  is 
attached  in  Appendix  G. 

2.  Reources  for  the  Bottom-Up  Review 

Resources  for  the  bottom-up  review  included  material  from  the  existing 
SPRDE/SE  certification  courses — SYS  201A  &  B  and  SYS  301 — and  IDA’s  content 
reviews  of  those  courses.  DAU  also  provided  IDA  with  a  number  of  additional  sources  to 
draw  from.  A  list  of  these  resources  is  shown  below: 

•  ACQ  101,201  A  and  B  TLOs 

•  IDA’s  content  reviews  and  reviews  by  “hooks”  of  SYS  201  A  and  B,  and  301 

•  Course  Student  Assessment  Plans  (CSAPs)  for  SYS  201  A  and  B  and  301 

•  “DAU’s  Systems  Engineering  Department  Revamping  SYS-301  Course,”  by 
Dr.  Martin  Falk  (included  in  Appendix  H) 

•  “SPRDE  Curriculum  Assessment,”  James  Childress,  Program  Director  SE  and 
T&E,  18  December  2000 

•  SPRDE  competencies  from  past  years 

Appendix  K-2  of  the  LMI  Workforce  Report  includes  systems  engineering-related 
competencies,  so  we  also  used  these  competencies  in  the  bottom-up  review  (these 
competencies  are  included  in  this  paper  in  Appendix  I).  Note  that  the  LMI  report  is  also 
listed  under  the  top-down  sources  as  well.  The  Future  Acquisition  and  Technical 
Workforce  Report,  published  in  early  2000,  contains  certain  assumptions  (called 
“trends”)  upon  which  its  list  of  competencies  is  based.  Before  using  the  LMI 
competencies,  IDA  wanted  to  validate,  change,  or  eliminate  those  assumptions  based  on 
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the  current  environment  to  determine  how  to  include  the  competencies  in  our  SPRDE/SE 
Duties  and  Tasks  list.  We  presented  the  LMI  Workforce  Report  Assumptions  to  the 
SPRDE/SE  FIPT  and  asked  them  to  review  the  assumptions.  Based  on  feedback  from  the 
SPRDE/SE  FIPT,  the  assumptions  were  modified  for  2004.  The  briefing  IDA  provided  to 
the  FIPT  along  with  the  FIPT’s  comments  is  included  in  Appendix  J. 

3.  Emphasis  on  Environmental,  Safety  and  Occupational  Health 

At  the  request  of  Mark  Schaeffer,  Director  of  Systems  Engineering,  IDA 
coordinated  with  Environmental,  Safety,  and  Occupational  Health  (ESOH)  subject  matter 
experts  to  ensure  that  ESOH-related  material  was  included  in  the  competencies  for  the 
SPRDE/SE  career  path.  Mr.  Schaeffer  is  the  chair  of  Acquisition  and  Technology  for 
Safety  on  the  Defense  Safety  Oversight  Council  (DSOC).  His  request  for  us  to  work  with 
the  ESOH  SMEs  came  out  of  an  initiative  to  revitalize  the  safety  message  by  improving 
communication  of  the  safety  problem.  ESOH  does  not  have  a  career  field/path  or  a 
Functional  Advisor  (although  they  have  fonned  an  ad  hoc  FIPT);  thus,  it  is  important  that 
the  ESOH  material  be  covered  in  the  existing  career  fields  and  paths.  It  was  suggested 
that  perhaps  some  ESOH  material  could  be  input  to  some  of  the  existing  case  studies 
used  in  the  SPRDE/SE  curriculum.  Other  activities  related  to  this  revitalization  include 
the  System  Safety  CLM  (through  the  SPRDE/SE  FIPT),  the  Corrosion  RDT,  and  the 
Corrosion  CLM  (developed  by  the  National  Association  of  Corrosion  Engineers,  or 
NACE). 

IDA  suggested  that  the  ESOH  SMEs  develop  a  list  of  ESOH-related  “Duties  and 
Tasks”  based  on  the  BCEFM  model,  just  as  we  were  doing  for  SPRDE/SE  in  general. 
The  ESOH  SMEs  attended  our  Duties  and  Tasks  development  meetings  and  provided  us 
with  a  list  of  ESOH-related  duties  and  tasks.  We  incorporated  it  into  our  own  list. 

4.  Developing  the  SPRDE/SE  Duties  and  Tasks 

Using  the  summary  materials  we  generated  from  the  sources  listed  above  (e.g., 
“Identified  Systems  Engineering  Best  Practices  or  Deficiencies,”  “Systems  Engineering 
Topics,”  and  IDA’s  content  reviews) — all  of  which  are  included  in  the  appendixes — IDA 
began  to  create  a  list  of  SPRDE/SE  duties  and  tasks.  We  reviewed  each  of  the  summaries, 
pulling  out  the  material  that  was  most  appropriate  for  today’s  systems  engineering 
environment  and  rewriting  the  material  as  needed  to  be  in  the  form  of  a  “duty  or  task.” 
The  authors  started  with  the  bottom-up  sources — the  existing  and  past  SPRDE/SE 
competencies,  TLOs,  and  ELOs.  After  sifting  through  those  resources,  we  next  went 
through  the  LMI  competencies,  again  selecting  those  appropriate  to  today’s  SPRDE/SE 
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workforce  for  the  SPRDE/SE  Duties  and  Tasks  list.  Best  practices  and  deficiencies  were 
next,  followed  by  the  systems  engineering  hooks  and  finishing  with  the  table  of  topics  to 
make  sure  that  none  of  the  important  systems  engineering  topics  was  left  out.  In  an  effort 
to  be  consistent  with  the  Systems  Engineering  chapter  of  the  Acquisition  Guidebook,  we 
organized  the  Duties  and  Tasks  list  according  to  the  sections  in  that  chapter  of  the 
Guidebook.  Some  material,  such  as  information  on  systems  engineering  by  phases,  was 
taken  directly  from  the  Systems  Engineering  chapter  of  the  Guidebook.  IDA  also 
organized  the  materials  by  certification  level,  putting  systems  engineering  theory  into 
Level  1,  systems  engineering  by  acquisition  phases  into  Level  2,  and  management  and 
leadership  information  into  Level  3.  IDA’s  list  of  SPRDE/SE  Duties  and  Tasks  can  be 
found  in  Appendix  K. 

B.  SPRDE/SE  PERFORMANCE  OBJECTIVES 

IDA  delivered  the  draft  list  of  SPRDE/SE  Duties  and  Tasks  to  the  Enterprise 
Development  office  on  19  May  and  we  met  with  DAU  representatives — John  Snoderly 
and  George  Prosnik — and  representatives  of  the  Enterprise  Development  office  shortly 
thereafter.  After  reviewing  the  Duties  and  Tasks,  John  Snoderly,  Program  Director  for 
SPRDE/SE  Curriculum,  and  George  Prosnik,  Director,  Engineering  and  Technology 
Center,  DAU  Curriculum  Development  Support  Center  (CDSC),  recommended  that  we 
develop  a  shorter,  more  succinct  product  from  the  Duties  and  Tasks.  They  indicated  that 
the  Duties  and  Tasks  in  their  current  form  might  overwhelm  the  SPRDE  FIPT. 

At  that  same  meeting,  we  collaboratively  developed  a  shorter  list  of  Duties  and 
Tasks  in  the  form  of  Performance  Objectives.  Taking  one  or  two  of  the  most 
encompassing  Duties  and  Tasks  from  each  subject  area,  we  rewrote  them  as  POs.  The 
POs,  from  27  May  2004  are  attached  in  Appendix  L.  The  DAU  representatives  took  the 
Duties  and  Tasks  indicating  that  the  detail  would  be  useful  for  them  in  developing  TLOs, 
ELOs,  and  course  material  for  the  certification  courses.  They  also  told  us  that  they  would 
do  a  gap  analysis  to  determine  the  extent  to  which  the  POs  we  developed  that  day  were 
covered  by  the  existing  course  material  in  SYS  201A  and  B  and  SYS  301. 

C.  SPRDE/SE  LEARNING  OUTCOMES 

When  the  FIPT  members  saw  the  POs  and  were  concerned  that  they  might  lack 
important  detail,  they  decided  to  expand  the  POs  to  include  more  specific  activities  by 
certification  level.  The  new  list  became  “Learning  Outcomes.”  At  that  1  July  FIPT 
meeting,  the  group  assigned  specific  individuals  to  expand  the  POs.  IDA  was  assigned  to 
expand  the  systems  engineering  processes,  while  others  were  assigned  other  sections  of 
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the  POs.  IDA  reconsidered  the  Duties  and  Tasks  we  developed  previously  in  the  area  of 
systems  engineering  processes,  and  submitted  that  list  divided  into  the  three  certification 
levels.  A  current  list  of  the  LOs,  dated  3 1  August  2004  can  be  found  in  Appendix  M.  This 
is  the  version  that  the  SPRDE/SE  FIPT  delivered  to  DAU. 
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VI.  NEXT  STEPS  AND  RECOMMENDATIONS 


A.  NEXT  STEPS 

At  this  time,  the  SPRDE/SE  FIPT  has  provided  Duties  and  Tasks,  Performance 
Objectives  (POs),  and,  most  recently,  Learning  Outcomes  (LOs)  to  DAU  as  guidance  for 
the  certification  courses.  DAU  is  in  the  midst  of  conducting  a  gap  analysis  to  determine 
which  of  the  LOs  are  covered  by  existing  SPRDE/SE  course  material.  In  the  period  to 
follow,  DAU  will  be  developing  TLOs,  ELOs,  and  course  material  to  support  the  LOs. 
Meanwhile,  in  addition  to  reviewing  DAU’s  SPRDE/SE  TLOs,  ELOs,  and  course 
material,  the  SPRDE  FIPT  will  begin  reviewing  the  systems  engineering-related  material 
in  the  other  career  fields/paths. 

1.  Reviewing  Systems  Engineering  Material  for  Other  Career  Fields/Paths 

Systems  engineering  material  is  primarily  covered  in  DAU’s  SPRDE/SE 
certification  courses,  but  it  is  important  for  other  career  fields/paths  as  well.  Everyone  in 
the  FIPT  agreed  that  reviewing  the  systems  engineering-related  material  in  the 
SPRDE/SE  path  was  the  first  step,  and  that  now  having  a  list  of  SPRDE/SE  duties  and 
tasks,  POs  and  LOs  would  permit  a  gap  analysis  or  mapping  to  detennine  whether  other 
non-SPRDE/SE  courses  should  cover  some  of  the  LOs.  Mr.  Schaeffer  requested  that 
SPRDE/SE  FIPT  review  the  systems  engineering-related  material  in  the  courses  outside 
the  SPRDE/SE  career  path  according  to  his  priorities.  He  grouped  the  courses  outside  the 
SPRDE/SE  career  path  into  two  “tiers.”  First  tier  courses  were  considered  higher  priority 
because  they  would  likely  have  a  greater  impact  on  systems  engineering  than  other 
courses.  Acquisition  (ACQ)  and  Program  Management  (PMT)  courses  are  considered 
“first  tier,”  while  review  of  other  courses  such  as  those  in  the  Production,  Quality,  and 
Manufacturing  (PQM)  career  field  were  considered  lower  priority  and  were  labeled 
“second  tier.”  The  list  of  “tiers”  can  be  found  in  Appendix  N. 

IDA  suggested  that  in  order  to  modify  course  material  from  another  career 
field/path,  we  first  needed  to  understand  how  the  courses  are  structured.  Unlike  the 
SPRDE/SE  courses,  not  all  of  the  PM  courses  are  based  on  published  TLOs  or  ELOs.  For 
example,  some  of  the  courses  intended  for  high-level  acquisition  personnel,  are  taught 
based  on  the  demands  and  questions  of  the  students.  To  help  the  Systems  Engineering 
office  better  understand  the  courses  they  are  trying  to  affect,  IDA  prepared  a  table 
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summarizing  the  PM  and  ACQ  courses  along  with  recommendations  about  how  they 
should  be  addressed.  It  is  attached  in  Appendix  O. 

The  Enterprise  Development  Office  recently  asked  IDA  to  begin  the  review  of 
systems  engineering-related  materials  in  other  career  fields/paths  by  determining  what 
SPRDE/SE  material  belongs  in  the  ACQ  101  and  201  TLOs.  We  recently  completed  a 
comparison  between  the  systems  engineering  policy  and  “hooks”  to  the  ACQ  101  and 
ACQ  201  TLOs  and  ELOs  and  proposed  revisions  to  the  ACQ  TLOs.  The  Enterprise 
Development  Office  will  send  the  proposed  TLO  revisions  to  the  Program  Management 
FIPT  for  consideration. 

2.  DAU  Course  Development 

DAU  is  currently  conducting  a  gap  analysis  by  comparing  the  SPRDE/SE  LOs  to 
the  existing  SPRDE/SE  certification  course  material.  After  finishing  the  gap  analysis, 
DAU  will  start  developing  TLOs  and  ELOs  based  on  the  LOs.  IDA  stressed  that  the  FIPT 
has  provided  a  large  amount  of  material  for  three  courses  and  suggested  that  we  keep 
track  of  material  currently  in  the  DAU  SPRDE/SE  courses  that  might  be  lost. 

B.  RECOMMENDATIONS 

After  participating  in  the  ongoing  review  of  SPRDE/SE  education  and  training, 
we  have  some  recommendations  for  the  SE  Office  and  the  SPRDE/SE  FIPT  as  they  move 
forward  with  the  remainder  of  the  review  or  in  their  future  reviews  of  the  DAU  material. 

Recommendation  1:  Conduct  a  more  thorough  requirements  analysis  of 
SPRDE/SE  acquisition  personnel  and  their  needs  for  SPRDE/SE  training.  In  the  course  of 
review,  we  did  not  have  the  time  to  do  as  much  of  the  requirements  development  process, 
one  of  the  most  important  processes  in  systems  engineering,  as  we  would  have  liked. 
Having  a  better  sense  of  the  requirements  would  have  led  to  more  complete  criteria  for 
reviewing  the  courses.  It  would  be  interesting,  for  example,  to  examine  the  areas  where 
systems  engineers  have  the  most  trouble  in  programs,  and  to  emphasize  those  areas  in  the 
SPRDE/SE  education  and  training.  We  attempted  to  do  this  through  previous  studies  but 
did  not  have  the  real-time  data  that  we  needed.  An  analysis  like  this  might  also  be  useful 
in  determining  whether  the  certification  courses  are  the  resources  that  need  update  and 
investment  or  if  courses  that  are  more  widely  available  on-demand  (such  as  CLMs)  might 
be  a  better  approach.  The  assessment  data  needs  to  feed  into  the  FIPT  process. 

Recommendation  2:  DAU  should  clarify  and  standardize  instructional 
terminology.  If,  rather  than  mandating  use  of  a  specific  process,  DAU  wants  to  ensure 
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that  each  of  the  FIPTs  can  be  flexible  in  their  management  of  the  career  fields/paths,  it 
would  be  helpful  if  DAU  defined  its  tenninology  generically  enough  to  allow  for  this 
variation.  While  the  instructional  terminology  definitions  in  Chapter  II  are  helpful  as  a 
starting  point  in  understanding  the  education  and  training  review  process,  the  use  of  the 
tenninology  is  confusing.  Although  the  FIPT  charter  indicates  that  “Competencies” 
should  be  developed/reviewed,  career  fields/paths  develop  LOs,  duties  and  tasks,  and  the 
like  instead  of  competencies,  hence  our  recommendation. 

Recommendation  3:  Define  requirements  carefully  up  front,  and  rigorously  follow 
systems  engineering  processes.  Our  process,  like  so  many  programs’  processes,  suffered 
from  requirements  changes  along  the  way.  For  example,  we  started  out  with  a  goal  to 
create  duties  and  tasks,  the  requirement  changed  to  develop  POs,  and  the  group  finally 
settled  on  LOs.  One  example  where  we  could  have  used  stronger  systems  engineering 
processes  was  in  configuration  management — specifically  tracking  decisions  made 
throughout  the  review  process.  Agendas  and  follow-up  minutes  are  always  developed  for 
the  SPRDE/SE  FIPT,  but  perhaps  this  practice  would  be  useful  for  the  smaller  meetings 
as  well.  Our  experience  indicated  that  decisions  made  at  one  meeting  that  necessitated 
follow-up  actions  were  sometimes  negated  in  future  meetings.  Another  example  relates  to 
the  configuration  control  of  the  review  documents.  The  education  and  training  review 
was  a  collaborative  effort  with  participation  from  many  people  and  organizations.  Given 
the  number  of  people  and  organizations  editing  documents,  we  may  need  to  use  more 
sophisticated  ways  of  tracking  documents’  versions  and  configuration.  Perhaps  we  could 
make  better  use  of  the  SE  CoP  workspace  provided  to  the  group  by  DAU  for  this 
purpose. 
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Appendix  A 

Wynne  Memorandum:  Policy  for  Systems  Engineering  in  DoD 


WYNNE  MEMO 


ACQUISITION 
TCCHNOCOCV 
AND  f  OrtHTICH 


THE  UNDER  SECRETARY  OF  DEFENSE 

3010  Ufc>tNfafc  HLNTAGON 
WASHINGTON,  DC  20301-3010 


FEB  20  2004 


MEMORANDUM  FOR:  SEE  DISTRIBUTION 


SUBJECT:  Polity  foi  Systems  Engineering  in  Do D 


Application  of  rigorous  systems  engineering  discipline  is  paramount  to  the 
Department’s  ability  to  meet  the  challenge  of  developing  and  maintaining  needed 
warfighting  capability.  This  is  especially  true  as  we  strive  to  integrate  increasingly 
complex  systems  in  a  family-of-systems,  system-of-systems,  net-centric  warfare  context. 
Systems  engineering  provides  the  integrating  technical  processes  to  define  and  balance 
system  pciful  malice,  cost,  schedule,  and  risk.  It  must  be  embedded  in  program  planning 
and  performed  across  the  entire  acquisition  life  cycle, 

Toward  that  end,  I  am  establishing  the  following  policy,  effective  immediately  and 
to  be  included  in  the  next  revision  of  the  DoD  5000  set  ics  acquisition  documents: 

Systems  Engineering  (SE).  All  programs  responding  to  a  capabilities  or 
requirements  document,  regardless  of  acquisition  category,  shall  apply  a 
robust  SE  approach  That  balances  total  system  performance  and  total 
ownership  costs  within  the  family-of-systems,  sysiems-of-systems  context. 

Programs  shall  develop  a  Systems  Engineering  Plan  (SEP.i  for  Milestone- 
Decision  Authority  (MDA)  approval  in  conjunction  with  each  Milestone 
review,  and  integrated  with  the  AcquisiUon  Strategy.  ITiis  plan  shall  describe 
the  program  s  overall  technical  approach,  including  processes,  icsuurces, 
metrics,  and  applicable  performance  incentives.  It  shall  also  detail  the  timing, 
conduct,  and  success  criteria  of  technical  reviews. 

In  suppori  of  the  above  policy,  the  Director,  Defense  Systems  shall: 

a.  Identify  the  requirement  for  a  SEP  in  DODI  5000  2,  and  provide  specific 
content  guidance  tailorable  by  the  MDA  in  the  Defense  Acquisition  Guidebook. 

b.  Assess  the  adequacy  of  current  Department-level  SE  related  policies, 
processes,  practices,  guidance,  tools,  and  education  and  training  and  recommend 
to  me  necessary  changes. 
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c.  Establish  a  senior-level  SE  forum  with  participation  from  the  Military 
Departments,  and  appropriate  defense  agencies,  as  a  means  to  collaborate  and 
leverage  activities  within  the  components  and  to  provide  a  forum  to 
institutionalize  SE  discipline  across  the  Department  A  goal  of  this  forum  will  be 
extending  the  SE  process  to  address  fainilv-of  systems,  system-of-systems 
capability-based  acquisition 

d.  For  programs  where  I  am  the  MDA.  review  each  piogram's  SEP  as  part 
ot  the  preparation  lor  Defense  Acquisition  Board  Milestone  Reviews  (DAB)  and 
other  acquisition  reviews,  provide  me  with  a  recommendation  on  the  program’s 
readiness  to  proceed  during  the  DAB  Together  with  other  members  of  the  OSD 
staff,  lead  program  support  assessments  to  identify  and  help  resolve  issues  to 
ensure  program  success. 

To  assist  in  these  efforts,  each  Component  Acquisition  Executive  and  defense 
agency  wuh  acquisition  responsibilities  will,  within  90  days,  piovide  the  Director 
Delensc  Systems  its  approach  and  recommendations  on  how  we  can  ensure  that 
application  oi  sound  systems  engineering  discipline  is  an  integral  pan  of  overall  program 
planning,  management,  and  execution  within  both  DoD  and  defense  industry.  Further  I 
uirecl  each  Component  Acquisition  Executive  and  those  defense  agencies  with 
acquisition  responsibilities  to  provide,  within  30  days,  a  (lag  officer  or  Senior  Executive 
Service-level  representative  to  participate  in  the  Director,  Defense  Systems-led  svstems 
engineering  forum.  The  fust  such  forum  will  be  held  within  60  days. 

1  need  your  assistance  to  ensure  we  drive  good  systems  engineering  processes  and 
practices  back  into  the  way  we  do  business.  We  can  accomplish  this  goal  by  establishing 
clear  policies,  reinvigorating  our  training,  developing  effective  tools,  and  using  and 
institutionalizing  best  practices,  applying  performance  incentives,  ami  making  systems 
engineering  an  imporlant  consideration  during  source  selections  and  throughout  contract 
execution.  Collectively  these  actions  will  rcinvigorate  our  acquisition  community  - 
including  our  industry  partners  -  thus  assuring  affordable,  supportable,  and  above  all 
capable  solutions  for  the  warfighter. 
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Appendix  B 

Lamartin  Memorandum:  Implementing  Systems  Engineering  Plans 

(SEP)  in  DoD 


LAMARTIN  SEP  MEMO 


ACGUTSITIOM, 
TECHNOLOGY 
AND  LOGISTICS 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON,  DC  ?030l  3000 


March  30.  2004 


MEMORANDUM  FOR:  SEE  DISTRIBUTION 

SUBJECT:  Implementing  Systems  Engineering  Plans  in  DoD  -  Interim  Guidance 


On  February  20. 2004.  the  Acting  I  Jndcr  Secretary'  of  Defense  for  Acquisition. 
Technology  and  Logistics  (IISDI A  l&l.))  took  a  major  step  to  rcinvigorutc  l)ol)  Systems 
Engineering  by  signing  into  policy  a  requirement  that  All  programs  responding  to  a 
capabilities  or  requirements  document . .  shall  develop  a  Systems  Engineering  Plan 
(SEP*  for  Milestone  Decision  Authority  (MDA)  approval  in  conjunction  with  each 
Milestone  review  ”  This  memorandum  pros  ides  interim  guidance  concerning  the 
purpose  and  content  of  these  plans.  I  look  forward  to  working  with  your  representative  to 
the  new  Systems  Engineering  Forum  to  capture  best  practices  and  mature  this  guidance 
over  tittle.  The  StP  will  be  addressed  more  completely  in  future  updates  to  the  Defense 
Acquisition  Guidebook. 

The  purpose  of  the  SEP  is  Ui  lay  out  a  plan  that  should  guide  all  technical  aspects 
of  an  acquisition  program.  Program  managers  should  establish  the  SEP  early  in  the 
program  definition  phase  and  update  it  at  each  subsequent  milestone.  It  is  intended  to  be 
a  living  document,  tailored  to  the  program,  and  a  roadmap  that  supports  program 
management  by  defining  comprehensive  systems  engineering  activities,  addressing  both 
government  and  contractortcchnir.nl  activities  and  responsibilities.  The  SF.P  describes 
the  program's  overall  technical  approach,  including  systems  engineering  processes; 
resources;  and  key  technical  tasks,  activ  ities,  and  events  along  with  their  metrics  and 
success  criteria.  Integration  or  linkage  with  uiliei  piogram  management  control  efforts 
such  as  integrated  muster  plans,  integrated  master  schedules,  technical  perlbmmncc 
measures,  and  earned  value  management  is  tundamcniul  In  successful  application. 

There  is  no  prescribed  format  for  the  SEP.  However,  it  should  address  how 
systems  engineering  will  support  the  translation  of  system  capability  needs  into  an 
effective,  suitable  product  that  is  sustainable  at  an  affordable  cost.  Specifically,  a  well- 
prepared  SEP  will  address  the  integration  of  the  technical  aspects  of  the  program  with  the 
overall  program  planning,  systems  engineering  activities,  and  c.sccutiou  tucking  to 
include: 
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•  I  he  systems  engineering  processes  Id  he  applied  in  the  program  (e.g..  from  a 
standard,  a  capability  maturity  model,  or  the  contractor’s  process).  Describe 
how  the  processes  will  be  implemented  and  how  they  will  be  tailored  to  meet 
individual  acquisition  phase  objectives.  Describe  how  Ihc  SI-  processes  will 
support  the  technical  und  programmatic  products  required  of  each  phase. 

•  The  system’s  technical  baseline  approach.  Describe  how  Ihc  technical  baseline 
w  ill  be  developed,  managed,  and  used  to  control  system  requirements,  design, 
integration,  verification,  and  validation.  Include  a  discussion  of  metrics  (e.g.. 
technical  performance  measures)  for  the  technical  effort  and  how  these  metrics 
will  be  used  to  measure  progress. 

•  Event-driven  timing,  conduct,  success  criteria,  and  expected  products  of 
technical  reviews;  and  how  technical  reviews  will  he  used  to  assess  technical 
maturity,  assess  technical  risk,  und  support  program  decisions,  SEP  updates 
shall  include  results  of  completed  technical  reviews. 

•  I  lie  integration  of  systems  engineering  into  the  program's  integrated  product 
teams  tIPTs).  Describe  how  systems  engineering  activities  w  ill  be  imcgiaied 
within  and  coordinated  across  IPTs,  how  the  ll’Ts  will  he  organized;  what  SE 
tools  they  will  employ;  and  their  resources,  staffing,  management  metrics,  and 
integration  mechanisms.  Describe  how  systems  engineering  activities  arc 
integrated  in  the  program’s  overall  integrated  schedules 

l  or  programs  where  the  USDfAT&L)  is  the  Milestone  Decision  Aulhonly 
(MDA).  components  shall  submit  the  SEP  to  me  at  least  30  days  before  the  scheduled 
Defense  Acquisition  Hoard  (DAB)  milestone  review.  My  stull  and  l  will  evaluate  each 
program's  SEP  in  preparation  Ibr  the  DAB  review  and  in  support  of  Defense  Systems' 
other  acquisition  nrul  assessment  support  activities  I  encourage  all  MDAs  to  take  similar 
actions. 

The  tc lei cnccd  SEP  policy  is  already  in  effect,  so  I  urge  you  to  distribute  this 
guidance  memorandum  to  your  Program  Executive  Officers.  Program  Managers,  and  or 
Systems  Commanders.  For  additional  clarification  or  guidance  on  SEP  tailoring,  please 
contact  Mr  Mark  Schaeffer,  Director,  Systems  Engineering.  (7(13)  605-7417. 
mark.schaefler  tf  osd.mil.  or  Mr.  Bob  Skalamcra,  Deputy  Director,  Systems  Engineering 
( Enterprise  Development),  (703)  605-2300,  ruhert  skalamcraio  usd  m:l 


Director.  Defense  Systems 
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Appendix  C 

CONTENT  REVIEW  OF  SYS  201 


CONTENT  REVIEW  OF  SYS  201A 


General 


•  Update  to  reflect  new  5000  and  3170  environment 

•  Replace  the  current  model  with  the  Vee  model  or  at  least  perform  a  closer  review  of  the  course  material  to  ensure 
that  the  characteristics  of  the  Vee  model  are  being  addressed 

—  Much  of  the  lesson  content  on  the  processes  associated  with  the  old  model  is  still  relevant — Requirements 
Analysis,  Functional  Analysis  and  Allocation,  and  Synthesis  (Physical  Solution) 

•  Need  a  good  scrub  of  the  content  to  be  sure  that  the  sections  really  add  value  in  terms  of  having  someone  walk 
away  with  a  better  sense  of  SE — matching  competencies  to  lessons 

•  In  accordance  with  IDA‘s  planned  approach  to  developing  competencies,  focus  first  on  the  minimum  set  of  things 
that  one  should  know  about  SE.  From  reviewing  the  courses,  it  seems  that  requirements  must  have  been 
promulgated  that  focused  on  the  specifics  of  a  particular  interest  area  and  that  a  coherent,  bigger-picture  view  of 
the  systems  engineering  process  became  dominated  by  these  more  restricted  topics. 

•  Decide  how  to  refer  to  the  idea  of  a  Systems  Engineering  Plan  when  the  SEP  acronym  is  already  associated  with 
the  systems  engineering  process  within  the  community 

—  Consider  the  development  of  a  new  lesson  that  covers  a  Systems  Engineering  Plan 

•  Resequence  the  lessons  logically. 

•  As  per  SPRDE/SE  FIPT  February  2004  decision,  revise  this  course  to  focus  on  the  Vees  and  the  phases  of  the 
acquisition  life  cycle. 
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Lesson 

Number 

Lesson  Name 

Review  Comments 

1 

Systems  Acquisition  Overview 

•  This  lesson,  particularly  the  portion  focused  on  Acquisition  Reform,  needs  to  be  revised 
given  the  release  of  the  new  5000  and  3170  documents.  In  general,  DAU  doesn’t  date  their 
references,  which  could  cause  confusion.  And  now  Acquisition  reform  is  being  reformed. 

2 

Integrated  Product  and  Process 
Development 

•  Need  to  stress  integration  of  IPTs  and  make  sure  there  is  no  SE  IPT. 

3 

Systems  Engineering 

•  This  lesson  is  obviously  based  on  the  old  SE  model.  Since  the  focus  now  is  a  Vee-model 
approach,  changes  need  to  be  made  to  this  section. 

4 

Technology  Development  and 
Insertion 

•  One  of  the  most  easy  to  follow  lessons  with  a  lot  of  good  information.  Needs  to  be  updated 
to  talk  about  Pre-Milestone  B  SE  activities  and  technology  development  and  insertion  in  the 
new  spiral  development  and  evolutionary  acquisition  environment. 

5 

Environmental,  Safety,  and 

Health 

•  Blurring  of  the  lines  between  the  content  of  a  course  to  teach  about  SE  and  a  more  general 
course  about  acquisition  and  the  various  requirements  and  regulations  that  pertain  to  it. 

This  may  be  a  result  of  the  past  competency  analyses  and  negotiations  among  courses 
and  FIPTs. 

•  Should  have  a  lesson  on  the  “ilities”  in  general  and  how  SE  integrates  their  considerations 
into  the  design  of  both  the  product  and  processes.  Then  ESOH  is  one  of  those  ilities. 

•  Snoderly  briefed  in  Dec  03  that  Exercise  5  was  revised  to  provide  clearer  instruction  to 
students. 

6 

Systems  Engineering  Planning 

•  This  lesson  touches  on  the  SEMP,  so  will  want  to  make  sure  that  it  is  addressing  the  types 
of  things  that  are  being  envisioned  for  the  Systems  Engineering  Plan.  It  will  warrant  some 
additional  material  than  that  which  is  currently  contained  in  this  lesson. 

7 

Requirements  Analysis 

•  References  the  MNS  and  ORD  as  inputs  to  Requirements  Analysis,  so  needs  to  be 
updated  to  reflect  the  ICD,  and  possibly  the  CDD 

8 

Functional  Analysis  and 

Allocation 

•  This  lesson  highlights  the  Functional  Architecture,  referring  to  it  as  the  primary  output  of 
Functional  Analysis. 

9 

Synthesis 

•  This  lesson  does  not  really  explain  enough  about  any  one  topic  and  topics  covered  aren’t 
necessarily  the  right  things  to  cover.  An  approach  like  Bob’s  that  highlights  reviews  as  they 
are  tied  to  the  acquisition  system  and  any  other  checklist-type  approach  is  the  better  way  to 
go.  How  do  we  ensure  that  taking  this  course,  especially  in  an  online  format,  is  actually 

Lesson 

Number 

Lesson  Name 

Review  Comments 

providing  what  is  needed?  Do  people  just  sort  of  read  through  this  material,  but  don’t 
necessarily  understand  why  it  is  important  or  how  one  would  use  it.  Need  to  see  the  test 
questions. 

•  Recursive  Nature  Technical  Data  Package  (TDP)  topic  area  text  doesn’t  seem  to  explain 
TDPs,  but  instead  talks  about  the  Verification  Loop 

•  Open  Systems  Analysis  topic  area  text  really  doesn’t  match  the  section  heading 

•  Simulation-Based  Acquisition  (SBA)  topic  area  probably  shouldn’t  refer  to  this  in  caps  or 
use  the  acronym — not  an  initiative  any  more 

•  Joint  Programs  (Joint  Modeling  and  Simulation  system  (JMASS),  Joint  Simulation  Systems 
(JSIMS),  and  Joint  Warfare  Simulation  (JWARS)  [Not  sure  that  all  of  these  are  still  active 
programs.  Either  JSIMS  or  JWARS  may  have  gone  away?] 

10 

Verification 

•  The  Systems  Engineering  Process  is  referred  to  in  this  course  as  SEP.  This  will  cause 
confusion  if  the  office  decides  to  make  the  SEMP  the  Systems  Engineering  Plan,  another 
SEP. 

•  “Open  Systems  Testing”--this  section  focuses  on  “conformance  testing”  rather  than  open 
systems  testing  as  the  section  heading  suggests. 

•  There  is  a  blurring  of  the  lines  between  verification  and  validation.  This  distinction  needs  to 
be  made  clear  and  should  follow  from  getting  away  from  the  old  499B  model,  which  didn’t 
include  Validation.  Bob’s  Vee  models  should  at  some  level. 

11 

Systems  Engineering  Process 
Outputs 

•  In  “Development  of  the  Specification  Tree  in  Relation  to  the  Life  Cycle  Phases,”  the  text 
doesn’t  really  highlight  the  Life  Cycle  phases  portion  of  this  section’s  title— is  more  focused 
on  the  specification  tree. 

12 

Work  Breakdown  Structure 

•  A  pretty  thorough  introduction  to  WBS. 

13 

Solicitations  and  Source 

Selection 

•  This  is  one  of  those  lessons  that  does  not  have  the  most  direct  SE  relevance.  Follow  up 
with  mappings  to  competencies  and  reviews  of  the  ACQ  courses. 

14 

Trade  Studies 

•  With  Bob  wanting  to  emphasize  the  need  for  trades  between  the  various  portions  of  the 
Vee-model,  need  to  take  a  closer  look  at  this  lesson  on  Trades  to  make  sure  that  it 
captures  what  is  necessary. 
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Lesson 

Number 

Lesson  Name 

Review  Comments 

15 

Cost  Containment 

•  This  lesson  focuses  predominately  on  CAIV. 

16 

Configuration  Management 

•  This  lesson  contains  information  on  baselines. 

•  Interface  Control  Document  (ICD)  is  discussed — now  that  acronym  stands  for  Initial 
Capabilities  Document  as  well. 

17 

Risk  Management 

•  This  lesson  still  references  the  old  5000,  so  will  need  to  be  revised  to  reflect  the  new  5000. 
Also,  in  this  lesson,  like  some  of  the  others,  the  flow  of  the  material  is  strange.  For 
example,  the  Risk  Management  Process  Steps  section  has  four  steps  interspersed  with  a 
number  of  other  items,  which  should  really  probably  be  handled  as  subs  to  those  steps. 

•  Sections  on  “Risk  Management  Policy  and  Procedures”  (as  per  the  contractor’s  corporate 
policy)  and  “Corporate  Policy  and  Procedures  Documents”--distinction  between  them  is  not 
clear. 

•  Sections  on  Risk  Management  appear  in  other  lessons  (CAIV,  Planning,  PBBE),  but  there 
is  no  clear  reason  why  the  duplication  exists  or  what  the  linkages  are  between  the  lessons. 

18 

Technical  Performance 
Measurement 

•  The  material  in  this  section  is  organized  strangely.  Reorganization  would  make  it  flow  more 
logically.  And  why  is  one  whole  lesson  devoted  to  this  topic?  Ranking  in  importance  from 
competencies? 

•  For  example,  sections  labeled  “Technical  Performance  Measurement  (TPM)”  and  “Concept 
of  TPMs”  appear  later  in  the  lesson  than  other  topics. 

19 

Technical  Reviews 

•  This  lesson  mentions  that  technical  reviews  are  performed  after  each  level  of  development. 
Bob’s  Vee  model,  however,  has  some  reviews  happening  at  different  points  in  the  Vee. 

This  is  an  inconsistency  that  will  have  to  be  dealt  with.  Need  to  make  sure  the  list  of  typical 
technical  reviews  covered  match  up  with  those  being  proposed  by  Bob.  Also  still  has  the 
reviews  linked  to  the  previous  acquisition  phases. 

•  John  Snoderly  briefed  in  Dec  03  that  they  revised  Exercise  19  for  the  on-line  portion,  but 
also  briefed  that  this  exercise  would  be  deleted  from  the  on-line  portion  (201  A)  and 
incorporated  into  the  capstone  exercise  in  201 B,  which  will  emphasize  technical  reviews. 

20 

Product  Improvement 

•  Material  in  this  lesson  doesn’t  really  warrant  its  own  separate  lesson.  It  could  be  just  as 
easily  folded  into  another  lesson  that  talks  about  how  SE  can  be  used  for  the  development 
of  new  systems  and  the  improvement  of  already  in  development  and  existing  systems. 

CONTENT  REVIEW  OF  SYS  201B 


The  material  in  the  Review  Resources  portion  of  SYS  20 IB  that  we  have  appears  to  be  the  same  as  the  SYS  201 A  course  material 
reviewed  above.  John  Snoderly  from  DAU  briefed  in  Dec  03  that  a  capstone  exercise  was  being  developed  for  implemention  in 
Jan  04  to — 

•  Tie  together  many  of  the  SYS  20  IB  topics  discussed  during  the  week  as  a  final  exercise 

•  Incorporate  and  emphasize  Technical  Reviews,  thus  eliminating  Exercise  19  from  the  on-line  version 

In  his  briefing  at  the  first  SPRDE/SE  FIPT  meeting,  John  Snoderly  discusssed  the  following  anticipated  changes: 

•  Including  a  threaded  exercise 

•  Becoming  a  “tools-based”  course  that  incorporates  software  tools  that  students  can  use  in  exercises  and  take  away  with 
them  on  CD 

•  Enhancing  lesson  VGs:  More  dynamic  and  built  upon  SYS  201 A  materials 
We  never  saw  any  VGs  or  any  pictures  at  all  for  SYS  201  A,  only  the  on-line  text. 
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REVIEW  OF  SYS  301 
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CONTENT  REVIEW  OF  SYS  301 


♦  Recurring  theme  in  SYS  301 :  Software  practices  often  cover  at  least  the  concepts  in  Bob’s  hooks.  Need  to  investigate  why  there  is 
so  much  separation  of  the  software  material  from  just  general  systems  material.  Probably  this  stems  from  the  days  when  the  SIS 
office  existed. 

♦  Lessons  need  a  more  logical  flow 

♦  As  per  SPRDE/SE  FIPT  February  decision,  Level  3  courses  should  focus  on  gaining  the  knowledge  to  lead  or  manage  systems 
engineering.  This  course  will  need  to  be  updated  accordingly. 


Lesson 

Number 

Lesson  Name 

Review  Comments 

1 

Acquisition  Policy  and  the 
Resource  Allocation  Process 

•  This  first  lesson  has  a  slide  dedicated  to  the  new  3170.01c  and  JCIDS  process.  Without 
being  in  the  classes,  it  is  difficult  to  know  how  much  of  the  process  is  covered.  But,  it  may  be 
helpful  to  include  more  about  JCIDS  and  what  the  functional  capabilities  boards  do  and  how 
it  ail  relates  to  the  integrated  architectures  PMs  are  being  asked  to  develop? 

•  Snoderly  briefed  in  Dec.  03  that  this  is  being  updated  to  reflect  Policy/PBBE  changes. 

2 

Architecture  and  Interoperability 
(from  TOC)/  Interoperability  and 
Architecture  (from  Lesson 
Assignment  Sheet) 

•  GIG  is  mentioned  on  a  slide,  but  maybe  it  should  be  further  explained?  Perhaps  it  is 
explained  elsewhere  (p.  106) 

•  One  of  the  slides  says  that  interoperability  is  “information  exchange  between  two  or  more 
parties.”  This  definition  of  interoperability  does  not  seem  to  be  broad  enough  for  SE.  Should 
interoperability  include  more  than  that?  (p.  107) 

•  This  lesson  discusses  the  Interoperability  KPP  and  Interface  Exchange  Requirements 
(lERs).  These  may  have  been  modified  or  renamed  recently,  and  if  so,  they  would  need  to 
be  modified  within  this  lesson  as  well. 

•  Snoderly  briefed  in  Dec.  03  that  this  is  being  updated  to  better  prepare  students  for  new 
case  study  in  Lesson  8  (?). 

3 

System  Analysis  and  Control 

Tools 

•  Configuration  management  is  included  as  a  general  topic,  not  specific  to  requirements,  for 
example 

•  499B-like  model  is  used  to  illustrate  the  detailed  systems  engineering  process  in  the  student 
reference  section 

D-2 


Lesson 

Number 

Lesson  Name 

Review  Comments 

•  There  is  a  systems  engineering  Vee  used  on  p.  150  relating  to  the  Acquisition  process.  One 
Vee  is  used  across  systems  integration  (SI)  and  system  demonstration  (labeled  SDD  in  the 
diagram).  Vees  will  now  be  focus  of  the  201  courses. 

•  Mention  of  technical  reviews  here  (p.  166) 

•  Case  Study 

•  Snoderly  briefed  in  Dec.  03  that  this  is  not  being  changed. 

4 

Integrated  Product  and  Process 
Development 

•  What  about  the  government  vs.  contractor  roles  in  the  Gov’t-LTI  organizational  structure 
used  by  programs  like  FCS,  where  there  are  government  and  contractor  co-leads  for  each 
of  the  IPTs.  Some  clarification  on  the  division  of  labor,  etc.  would  be  helpful.  Issue  of  where 
the  responsibility  and  funding  lie. 

•  Systems  Engineering  Integration  of  Program  IPTs  should  be  highlighted  and  scrubbed  to  be 
sure  there  are  no  SE  IPTs. 

•  Mars  Rover  Exercise 

•  Snoderly  briefed  in  Dec.  03  that  this  is  not  being  changed. 

5 

Technology  Management 

•  This  lesson  could  be  appropriate  beyond  just  SPRDE  because  it  explains  the  process  and 
policies,  but  does  not  seem  to  be  specific  to  SE. 

•  Snoderly  briefed  in  Dec.  03  that  Huntsville  is  incorporating  new  case  focusing  on 

Technology  Transition. 

6-01 

DoD  Software  Acquisition 

Planning  and  Strategy  (from 

TOC)/  The  DoD  Software 
Acquisition  Management 
Environment  (from  Lesson 
Assignment  Sheet) 

•  Should  be  scrubbed  to  see  what  is  software-specific  or  broadened  to  talk  about  system 
Acquisition  and  Strategy  ... 

•  CMM  is  mentioned  in  a  teaching  note  with  respect  to  process  maturity.  Why  isn’t  CMMI 
mentioned  instead?  It  is  mentioned  in  a  later  lesson. 

6-02 

DoD  Software  Acquisition 

Control  and  Methodology  (from 
TOC)/  SW  Engineering  Process 
(from  Lesson  Assignment  Sheet) 

•  Discussing  the  different  software  development  paradigms  is  included  as  one  of  the  ELOs.  It 
would  be  nice  if  the  lesson  could  get  more  into  the  advantages  and  disadvantages  of  the 
different  development  paradigms — which  ones  are  most  appropriate  when  (there  is  a  little  bit 
of  this  in  the  next  lesson). 

•  499B-like  model  used  to  describe  the  systems  engineering  process 
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Lesson 

Number 

Lesson  Name 

Review  Comments 

6-03 

DoD  Software  Acquisition 

Planning  and  Strategy  (from 

TOC)/  SW  Best  Practices  (from 
Lesson  Assignment  Sheet) 

•  The  need  for  a  “Requirements  Management  Plan”  is  mentioned  in  the  Implementation 
Guidelines  under  one  of  the  recommended  best  practices  in  this  lesson:  Manage  and  Trace 
Requirements  (p.  370) 

•  The  Defense  Science  Board  Task  Force  Study  on  Software  (Nov  2000)  also  mentions 
requirements  management:  “technical  and  management  practices  for  better  requirements 
management  were  described  and  recommended  long  ago...  these  practices  are  the 
hallmark  of  commercial  best  practice  but  they  remain  largely  underutilized  in  the  acquisition 
and  development  of  defense  software.”  (p.  415) 

•  Some  of  the  best  practices  listed  in  this  chapter  apply  beyond  software  (e.g.,  independent, 
expert  reviews  suggested  in  the  DSB  study) 

6-04 

Software  Management  Exercise 
(Case  Scenario  and  Workshop) 

•  Case  Study 

•  Snoderly  briefed  in  Dec.  03  that  there  is  now  only  one  Software  Management  lesson,  not  the 

4  shown  in  the  course  material  reviewed  here.  Its  focus  is  metrics  and  best  practices  and 
the  case  study  is  the  same. 

7 

Concept  and  Technology 
Development 

•  Should  this  lesson  exactly  match  the  acquisition  phases?  This  lesson  relates  to  both  1) 
Concept  Refinement  and  2)  Technology  Development 

•  Case  Study 

8 

Environmental,  Safety,  and 
Occupational  Health. 

•  Case  Study 

•  Snoderly  briefed  in  Dec.  03  that  this  lesson  is  being  combined  with  International  lesson 
below  and  being  folded  into  new  lesson  titled  “Transition  to  Systems  Acquisition.” 

9 

Benefits  and  Challenges  of 
International  Cooperation 

•  No  lesson  comments 

•  See  above 

10 

System  Definition 

•  499B-like  model  used  as  the  SE  Process  Model  in  slides 

•  Should  this  lesson  map  exactly  to  an  acquisition  phase?  It  relates  to  System  Development 
and  Demonstration 

•  Case  Study 

•  Snoderly  briefed  in  Dec.  03  that  System  Defense  Case  Study  was  shortened  up  a  bit.  Was 
that  this? 
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11 

Design,  Fabrication,  and  Test 

•  Should  this  lesson  map  exactly  to  an  acquisition  phase?  Like  the  previous  lesson,  it  relates 
to  System  Development  and  Demonstration. 

•  Snoderly  briefed  in  Dec.  03  that  this  lesson  received  minor  updates. 

•  Exercise/  Case  Study 

12 

Current  Events  and  Issues 

•  The  idea  of  dedicating  a  lesson  to  sharing  individual  experience  is  a  good  one.  The 
students’  experience  and  examples  should  provide  additional  case  study-like  discussions. 

•  Snoderly  briefed  in  Dec.  03  that  this  lesson  was  unchanged. 

13 

Production:  IPPD  and  the 

Paladin  Enterprise 

•  Browsing  through  the  case  study,  the  499B  model  is  shown  a  number  of  times.  This  should 
be  replaced  with  the  Vee. 

•  Should  this  lesson  map  exactly  to  an  acquisition  phase?  It  relates  to  a  combination  of  two 
phases:  Production  and  Deployment  and  Operations  and  Support. 

•  Case  Study 

•  Snoderly  briefed  in  Dec.  03  that  this  lesson  was  unchanged. 

14 

Deployment,  Operations,  and 
Support 

•  Case  Study 

•  Should  this  lesson  map  exactly  to  an  acquisition  phase?  It  relates  to  a  combination  of  two 
phases:  Production  and  Deployment  and  Operations  and  Support. 

•  Snoderly  briefed  in  Dec.  03  that  DOS  was  cancelled.  Is  there  any  concern  about  this 
information  having  been  lost? 

15 

Improvements  to  Existing 

Weapon  Systems 

•  The  title  of  this  lesson,  the  TLO  and  one  of  the  ELOs  is  specific  to  weapon  systems.  Does 
that  make  sense?  Shouldn’t  the  lesson  be  broader?  There  are  more  kinds  of  systems  than 
just  weapon  systems.  Maybe  rename  to  something  about  Evolutionary  acquisition  and  spiral 
development? 

•  The  differences  between  evolutionary  acquisition  and  spiral  and  incremental  development 
don’t  seem  to  be  well  understood  in  the  community,  so  maybe  this  lesson  is  a  way  to  clarify 
that  point.  Could  there  be  examples  of  both  spiral  and  incremental  approaches  to  show  how 
the  two  are  different  and  where  each  is  appropriate?  Maybe  a  case  study  that  asks  the 
student  to  pick  a  development  approach  (which  could  include  the  two  of  these)? 

•  Snoderly  briefed  in  Dec.  03  that  Product  Improvement  was  cancelled. 
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16 

Modeling  and  Simulation 

•  Page  1 94  includes  some  other  organizations’  versions  of  SBA  (e.g.,  SMART,  SEBA).  The 
course  may  already  cover  this,  but  it  would  be  helpful  to  understand  the  differences  between 
these  different  flavors  of  SBA,  if  they  still  exist. 

•  Snoderly  briefed  in  Dec.  03  that  this  lesson  was  shortened  up  a  bit  to  reflect  true  content. 

17 

Professional  Ethics  Case 

•  Case  Study 

Spectrum  Management 

•  Snoderly  briefed  in  Dec.  03  that  this  is  a  new  one-hour  lesson  whose  content  is  being 
worked  by  DoD  Spectrum  Management  people. 

Evolutionary  Acquisition 

Capstone  Case 

•  Snoderly  briefed  in  Dec.  03  that  this  capstone  lesson  will  have  6  case  studies  dealing  with 
different  aspects  of  EA/product  improvements/OPS  support/etc. 
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Systems  Engineering  Strategy 

This  hook  is  not  explicitly  called  out  in  any  of  the 
SYS201  A  &  B  lesson  materials.  [Unless  this  is  really 
something  that  could  possibly  be  covered  under 

Lesson  6:  Systems  Engineering  Planning  in  SYS  201 
A?] 

In  most  instances,  “strategy”  is  used  to  refer  to  either 
an  Acquisition  or  S&T  strateqv.  In  Lesson  6:  Systems 
Engineering  Planning,  however,  there  were  several 
quotes  related  to  what  a  contractor’s  SEMP  should 
include  that  may  start  to  get  at  what  the  office  is 
looking  for  with  this  hook: 

•  “A  technical  strategy  description  that  ties  the 
engineering  level  effort  to  the  higher-level 
management  planning.” 

•  “A  description  of  how  the  SEP  will  be  tailored  and 
structured  to  complete  the  objectives  stated  in  this 
strategy.” 

•  “A  resource  plan  that  identifies  the  estimated 
funding  and  schedule  necessary  to  achieve  the 
strategy.” 

•  No  mention  of  a  Systems  Engineering  Strategy 
exactly,  but  there  are  a  couple  of  lessons  in  SYS 

301  with  "software  acquisition  planning  and  strategy" 
in  the  title  (specifically,  the  TOC  title,  not  the  actual 
lesson  title). 

Upfront  and  early  addressing 
of  SE  (all  acquisition  phases, 
not  just  SDD) 

Refers  to  the  application  of  SE  throughout  in  terms  of 
a  program,  which  probably  is  meant  to  entail  an 
acquisition  program.  If  this  is  the  case,  then  SYS 

201 A  &  B  do  not  explicitly  address  the  use  of  SE  pre- 
Milestone  B.  Perhaps  the  closest  that  it  gets  to 
addressinq  the  pre-Milestone  B  part  is  throuqh  Lesson 

4:  Technology  Development  and  Insertion  in  SYS201 

A,  which  acknowledges  technology  development 
programs  and  the  insertion  of  technologies. 

In  a  search  for  “early”  in  SYS  201  B:  Review 
Resources:  This  is  a  confusing  one,  because  for  the 

•  Some  of  the  lessons  within  this  course  are  divided 
roughly  by  acquisition  phase.  For  example,  Lesson 

5:  Technology  Management,  Lesson  7:  Concept  and 
Technology  Development,  Lesson  10:  System 
Definition,  Lesson  11:  Design  Fabrication  and  Test, 
Lesson  13:  Production,  Lesson  14:  Deployment, 
Operations  and  Support.  Perhaps  the  lessons 
should  be  divided  up  specifically  according  to  the 

5000  phases?  The  downside  of  this,  of  course,  is 
that  if  the  phases  change,  as  they  could  do,  a 
rewrite  of  the  lesson  would  be  needed. 
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most  part  when  they  refer  to  early  in  a  program,  it 
means  after  Milestone  B.  Other  times,  however,  you 
get  the  impression  that  early  means  prior  to  this,  but 
not  sure  who  has  purview  at  that  time  since  the 
program  hasn’t  been  initiated  yet.  The  following  are 
some  instances  where  pre-Milestone  B  does  come  up: 

•  Lesson  5:  Environmental  Safety  and  Health 
Considerations:  “The  best  time  to  integrate  ESOH 
considerations  into  the  SEP  is  early  in  the  life  cycle, 
during  Concept  Exploration,  Technology 
Development,  and  System  Integration  work 
activities.  It  is  easier  and  less  costly  to  make 
changes  to  hardware.  For  most  programs,  the  PM 
prepares  a  PESHE  by  milestone  B.” 

•  In  Lesson  6:  Systems  Enqineerinq  Planninq,  a  pre- 
Milestone  B  activity  is  mentioned  related  to  T&E 

•  Lesson  9:  Synthesis:  “Several  alternative  hardware 
and  software  configurations  must  be  developed  to 
satisfy  the  requirements  of  the  Functional 
Architecture.  Early  on  in  the  life  cycle  (before 
Milestone  A  and  the  concept  exploration  work 
effort),  these  alternatives  actually  represent 
systems  and  are  used  to  develop  preliminary 
concepts  for  the  system’s  design.” 

•  Lesson  12:  Work  Breakdown  Structure:  “Thouqh 
the  Concept  Refinement  and  Technical 

Development  phases,  WBSs  are  usually  in  an  early 
stage  of  development.  It  is  not  until  the  system 
integration  work  effort  in  the  System  Development 
and  Demonstration  phase  that  the  system  is 
described  by  the  System  Specification  and  the 

WBS  expands  to  include  lower  levels.  By  the  end  of 
the  Production  and  Deployment  phase  the  WBS  is 
fully  defined  to  its  lowest  elements.” 

•  The  lessons  that  are  roughly  divided  by  specific 
acquisition  phases  (e.g.,  Concept  and  Technology 
Development;  System  Definition;  and  Design, 
Fabrication  and  Test;  Production:  IPPD  and  the 
Paladin  Enterprise;  and  Deployment,  Operation  and 
Support)  have  TLOs  that  include  how  systems 
engineering  is  used  within  that  phase. 

•  Lesson  5:  Technology  Management  is  more  about 
the  DoD  S&T  process  in  general  than  specifically 
what  systems  engineers  need  to  be  doing  in  the 

S&T  phase.  Needs  to  focus  more  on  SPRDE  SE 
role. 

•  The  TLO  for  Lesson  7:  Concept  and  Technology 
Development  is  “to  evaluate  the  effective  execution 
of  the  entire  Concept  and  Technology  Development 
phase  using  a  systems  engineering  approach.”  So 
the  idea  of  performing  SE  from  the  start  of  the 
program  is  included  here.  There  is  also  a  slide  (p. 

608)  titled  “Systems  Engineering  in  Concept 
Exploration.”  “Acquisition  Strategy  Elements”  are 
also  included  here,  such  as  Open  System 

Objectives;  Cost,  Schedule,  and  Performance  Risk 
Management,  CAIV,  etc. 
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•  While  discussinq  CAIV  in  Lesson  15:  Cost 
Containment,  it  is  mentioned  that  “aggressive  yet 
realistic  cost  objectives”  should  be  established  in 
the  program  and  that  warfighter’s  should  be 
involved,  early  and  continuously,  in  establishing 
and  modifying  goals  throughout  a  program.  Also  in 
discussing  TOC,  it  seems  to  indicate  pre-Milestone 

B  work  when  it  states:  “TOC  objectives  should  be 
addressed  early  in  the  design  process.  This  allows 
performance  objectives  to  be  developed  that  are 
achievable  and  affordable  based  on  actual 
development  and  additional  analysis  during  System 
Development  and  Demonstration  (SD&D).”  Further, 
“Additional  cost/performance/schedule  trades  as 
well  as  specific  cost  reduction  studies  and  actions 
may  be  necessary  early  in  the  acquisition  process 
to  achieve  TOC  reductions.  This  may  require 
additional  up-front  funding.” 

•  Lesson  17:  Risk  Manaqement  talks  about  the  need 
for  “initial  system-level  risk  assessments”  during 
the  Concept  and  Development  phase 

Lead  systems  engineer 

assigned  at  program  inception 

This  hook  is  not  explicitly  called  out  in  any  of  the 
SYS201  A  &  B  lesson  materials 

In  a  search  for  “systems  engineer”  in  SYS  201  B: 
Review  Resources:  The  term  lead  systems  engineer 
does  not  appear.  A  systems  engineer  comes  up 
periodically  throughout  the  text.  In  Lesson  13: 
Solicitations/Source  Selections  it  states  that  the 
systems  engineer  is  “to  support  the  development  and 
maintenance  of  the  business  contract.”  If  the  systems 
engineer  is  supposed  to  assist  in  the  development  of 
a  business  contract,  it  would  seem  that  that  individual 
would  have  to  be  on  board  since  program  inception 
(Milestone  B),  if  not  before. 

•  Haven't  seen  specific  reference  about  the 
appointment  of  a  lead  systems  engineer.  But  in 
doing  a  terms  search  through  the  lessons,  there  are 
references  to  the  lead  of  the  systems  engineering 

IPT.  Some  of  the  case  studies  place  the  student  in 
the  role  of  SE-IPT  lead.  These  need  to  be  taken  out 
and  emphasis  placed  on  SE  integration  of  IPTs. 
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Acquisition  components  must 
maintain  a  robust  systems 
engineering  organization 

This  hook  is  not  explicitly  called  out  in  any  of  the 
SYS201  A  &  B  lesson  materials 

•  No  specific  references  to  maintaining  robust  systems 
engineering  organization. 

Lead  Engineer  is  jointly 
accountable  to  PM  and 
functional  leadership 

This  hook  does  not  appear  to  have  been  explicitly 
called  out  in  any  of  the  SYS201  A  &  B  lesson 
materials. 

•  No  specific  references  to  the  lead  engineer  jointly 
accountable  to  PM  and  functional  leadership.  These 
kinds  of  issues  are  addressed  in  the  IPPD 

Handbook  and  should  be  put  in  the  Guide  as  well. 

SE  spans  IPT  (not  located  in 
an  “SE  IPT”) 

Lesson  2  of  SYS201  A  &  B  is  Integrated  Product  and 
Process  Development  and  includes  topics  such  as 

IPPD  Concepts,  IPPD  Relationships  and  IPT 

Concepts.  There  is  also  a  section  under  IPPD 

Concepts  titled  “Relationship  of  Systems  Engineering 
to  IPPD,”  which  includes  the  following  statements: 

“IPPD  is  a  management  technique  that  integrates 
systems  acquisition  functions;”  and  “Systems 
engineering,  on  the  other  hand,  is  the 
structured/disciplined  approach  that  addresses  the 
technical  aspects  of  a  program  within  an  IPPD 
framework.”  Given  the  way  that  IPTs  have  been 
misused  at  times,  however,  perhaps  it  would  be  worth 
explicitly  saying  something  to  the  effect  of  this  hook 
somewhere  in  this  lesson. 

•  References  refer  to  the  SE  IPT,  as  opposed  to  SE 
spanning  the  IPTs.  See  the  comment  about  the 
“Lead  systems  engineering  assigned  at  program 
inception.” 

Systems  Engineering  Plan 
(SEP)  established  early  in  the 
program  definition  and  updated 
continuously  as  the  program 
matures,  through  system 
operations  and  support 

Lesson  6:  Systems  Enqineerinq  Planninq  in  SYS201A 
addresses  the  following  major  topics:  Program  vs. 
Technical  Planning,  Control  Criteria  and  Metrics, 

Event  Criteria  vs.  Time-based  Accomplishment 
Planning,  Influence  on  System  Design,  and  Contractor 
efforts.  It  talks  at  least  a  little  about  the  SEMP  under 
the  section  on  Technical  Plans  and  also  provides  a 
little  more  information  on  why  one  would  want 
systems  engineering  plans.  No  real  information  about 
the  timing  of  the  development  of  a  SEP  or  a  schedule 
for  its  continuous  update  throughout  the  course  of  the 

•  SEMP  (Systems  Engineering  Management  Plan)  is 
mentioned  in  Lesson  6-03:  SW  Best  Practices  (p. 
370-371)  as  a  place  where  requirements  traceability 
to  address  system,  hardware,  and  software  and 
process  should  be  defined.  Early  and  continuously 
updated  not  specified. 

•  Note  that  the  acronym  SEP  is  used  to  mean 
“Systems  Engineering  Process”  in  Lesson  3:  System 
Analysis  and  Control  Tools  on  p.  149,  as  well  as  on 
p.  186  and  p.  202  (Lesson  14:  Deployment, 

Operations  and  Support).  There  are  several 
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program 

mentions  of  various  plans  (risk  management, 
software  development,  etc). 

•  Individual  plans  (test  plans,  project  plans, 

management  plans,  etc.)  are  discussed  throughout 
the  lessons. 

SEP  description  of  systems 
integration  on  the  program 
IPTs,  including  resources, 
staffing,  management  metrics, 
and  integration  mechanisms 

In  addition  to  the  information  described  in  the  hook 
above,  Lesson  6’s  section  on  Control  Criteria  and 
Metrics  includes  information  on  metrics  in 
management  and  product  metrics,  as  well  as  MOEs 
and  MOPs.  Little  if  any  specific  information  however 
that  describes  the  SEP  and  a  description  of  systems 
integration  on  program  IPTs  to  include  resources, 
staffing  and  integration  mechanisms. 

•  Chapter  6-01 :  The  DoD  Software  Acquisition 
Management  Environment  includes  a  passage  about 
how  systems  integration  (specifically  software  to 
software,  software  to  hardware,  and  software  to 
systems)  needs  to  be  included  in  the  master 
timeline,  but  have  not  seen  specific  reference  that  it 
needs  to  go  into  a  SEP. 

•  There  is  an  entire  lesson  on  “System  Analysis  and 
Control  Tools”  (Lesson  3),  but  have  not  seen 
specific  mention  of  the  need  to  document 
management  metrics  in  the  SEP. 

•  As  mentioned  in  the  comments  for  the  previous 
hook,  there  are  individual  plans  mentioned 
throughout  the  lesson  that  may  contain  the  topics  in 
this  “hook”,  but  no  specific  mention  of  a  SEP. 

Independent  Subject  Matter 
Experts  on  SETRs  (Systems 
Engineering  Technical  Review) 

Lesson  19:  Technical  Reviews  does  not  specifically 
mention  the  issue  of  those  involved  in  technical 
reviews  being  independent.  However,  under  “General 
Principles”  in  the  Purpose  of  Technical  Reviews 
section,  it  states  that  review  participants  are 
designated  and  “should  include  all  stakeholders  who 
are  knowledgeable  of  the  area  concerning  the 
technical  review  (includes  subcontractors,  vendors, 
and  suppliers). 

•  Noticed  that  there  is  specific  lesson  on  Technical 
Reviews  in  Sys  201 .  Information  about  technical 
reviews  appears  to  be  spread  out  in  various  lessons 
of  301 — should  it  be  combined  into  one  lesson  for 

301,  or  is  it  adequately  covered  in  201? 

•  Lesson  6-01 :  The  DoD  Software  Acquisition 
Environment.  The  words  “USD(S&T)  independent 
‘expert  review’”  are  included  in  a  slide  referring  to 

DoD  SAM  Key  Policies  from  the  Defense  Acquisition 
Guidebook  (p.  317). 

•  Lesson  6-03:  SW  Best  Practices  includes  the 

Defense  Science  Board  Task  Force  on  Defense 
Software  report  (or  at  least  excerpts  of  it).  One  of  the 
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best  practices  in  this  report  is  to  “Initiate 

Independent  Expert  Reviews.”  The  DSB 
recommended  institutionalizing  them  for  DoD  ACAT 
l-lll  software  intensive  programs.  The  report 
recommends  that  the  IER  team  be  “a  small  group  of 
professionals  with  the  appropriate  mix  of 
experience...  should  be  drawn  from  government, 
academia,  and  contractor  resources...  no  IER  team 
should  be  directly  involved  in  the  program.”  (p.  20- 
21)  Sample  topics  and  agenda  are  included. 

•  Lesson  13:  IPPD  &  The  Paladin  Enterprise  says  that 
“The  nature  of  technical  reviews  is  confirmation;  they 
validate  and  coordinate  activities  which  preceded 
them.  For  example,  no  new  information  should  be 
presented  in  a  technical  review.  All  pertinent 
information  must  have  been  reviewed  and 
considered  adequate  by  the  government  prior  to 
review...”  There  are  also  brief  discussions  of 
Production  Phase  Evaluations  and  Technical 

Reviews/ Audits/Tests  such  as  PCA,  PAT&E 
(Production,  Acceptance,  Test  and  Evaluation), 
FOT&E  (Follow-up  Operational  Test  and 

Evaluation).  Do  these  technical  reviews  also  need  to 
have  SMEs,  independent  reviewers,  etc.  even 
though  they  do  not  appear  to  be  included  in  the 

5000  timeline? 

SETR  chaired  by  an 

independent  technical  authority 

This  hook  is  not  explicitly  called  out  in  any  of  the 
SYS201  A  &  B  lesson  materials 

•  See  comment  about  “Initiate  Independent  Open 
Reviews”  in  Lesson  6-03  in  previous  hook. 

•  Lesson  07:  Concept  and  Technology  Development 
includes  a  case-based  lesson  and  content  about 
preparing  for  Technology  Development  Decision 
Reviews  and  Milestone  B.  One  of  the  ELOs  is  “to 
evaluate  the  results  of  Concept  and  Technology 
Development  Phase  concept  analyses  by  using 
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technical  reviews”  but  there  does  not  appear  to  be 
any  guidance  about  who  conducts  the  reviews,  if 
there  are  SMEs  involved,  or  whether  or  not  the 
technical  reviews  are  required  contractually. 

•  Lesson  10:  System  Definition  includes  “Holding 
Technical  Reviews”  as  one  the  SE  Activities  within 
the  Synthesis  phase  of  the  SE  Process  Model 
(would  it  be  more  appropriate  to  use  the  Vee  here 
instead?).  There  are  specific  questions  listed  for 
System  Requirements  Reviews  and  System 

Functional  Reviews  (one  slide  for  each  review)  but 
again,  no  mention  of  who  does  the  review,  etc. 

SETRs  addressed  in  and 
required  by  contractual 

documents 

Again  under  “General  Principles”  in  the  Purpose  of 
Technical  Reviews  section  of  Lesson  19:  Technical 
Reviews  mentions  “the  contractor  having  the 
contractual  responsibility  for  conducting  the  reviews 
and  having  all  the  data,  models  and  equipment  at  their 
site 

•  Lesson  03:  System  Analysis  and  Control  Tools 
describes  Technical  Performance  Measures,  and 
how  they  should  be  tracked  by  the  contractor  and 
reported  to  the  government  contractually.  There  is 
also  mention  of  the  technical  reviews  in  that  TPMs 
should  be  reviewed  during  those  reviews. 

•  Integrated  Baseline  Reviews  are  mentioned  on  a 
slide  under  the  subtitle  “Contract  Approach”  on  p. 

612  of  Lesson  7:  Concept  and  Technology 
Development. 

Tailored  technical  reviews 

The  issue  of  tailoring  is  not  specifically  mentioned,  but 
SYS201  A  contains  an  entire  lesson,  Lesson  19: 
Technical  Reviews,  devoted  to  this  topic.  The  major 
subject  headers  for  this  lesson  include:  Purpose  of 
Technical  Reviews,  Types  of  Technical  Reviews  and 
Configuration  Baselines  and  Technical  Reviews.  The 
specific  reviews  that  are  discussed  are  as  follows: 
Alternative  Concept  Review  (ACR),  System 
Requirements  Review  (SRR),  System  Functional 
Review  (SFR),  Software  Specification  Review  (SSR), 
Preliminary  Design  Review  (PDR),  Critical  Design 
Review  (CDR),  Production  Readiness  Review  (PRR), 

•  The  “opportunity  for  tailoring”  reviews  is  raised  on  a 
series  of  charts  in  lesson  6-02:  SW  Engineering 
Process  (p.  348-350)  describing  the  software 
engineering  process. 
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Test  Readiness  Review  (TRR),  and  System 

Verification  Review  (SVR). 

“Tailored  Technical  Reviews”  does  not  appear  in  SYS 
201  B:  Review  Resource 

Technical  baselines  approved 
at  reviews 

The  relationship  between  baselines  and  the  reviews 
is  established  in  the  Configuration  Baselines  and 
Technical  Reviews  section  of  SYS201  A’s  Lesson  19: 
Technical  Reviews.  In  particular  several  of  these 
relationships  are  highlighted:  System  Functional 

Review  and  the  functional  baseline,  between 
Preliminary  Design  Review  and  Critical  Design 

Review  and  the  allocated  baseline,  and  a  “Physical 
Configuration  Audit  performed  on  first  system 
manufactured  on  FRP  based  on  a  detail  specification 
vs.,  the  actual  system,  resulting  in  a  product 
baseline.”  “Technical  baselines”  does  not  appear  in 

SYS  201  B:  Review  Resources 

•  “Technical  reviews”  are  only  mentioned  twice  in  the 
301  course 

•  There  is  mention  of  technical  reviews  as  “quality 
gates”  (Lesson  6-01 :  The  DoD  Software  Acquisition 
Management  Environment)  on  p.  308:  “PMs  should 
approach  the  formal  review  as  a  checkpoint  for 
determining  whether  or  not  the  project  is  ready  to 
proceed  to  the  next  phase.” 

•  The  same  slides  that  describe  the  “opportunities  for 
tailoring”  reviews  also  discuss  baselines  as  well.  For 
example,  ASR,  SRR,  SDR  and  SFR  are  associated 
with  the  Functional  Baseline;  SSR  and  PDR  are 
associated  with  the  Allocated  Baseline;  and  CDR, 
TRR,  SVR  and  PCA  are  associated  with  the  Product 
Baselines  (p.  348-350).  As  with  so  many  of  the  other 
hooks,  this  should  also  be  covered  in  more  general 
lessons  that  are  not  focused  only  on  software. 

Technical  reviews  are  event 
driven  vice  schedule  driven 

There  is  a  specific  statement  to  this  effect  under 
“General  Principles”  in  the  Purpose  of  Technical 
Reviews  section  of  SYS201  A’s  Lesson  19:  Technical 
Review 

•  There  is  mention  of  event-based  technical  reviews 
on  p.  166  of  Lesson  3:  System  Analysis  and  Control 
Tools 

Systems  Engineering  in  Total 
Life  Cycle  Systems 

Management 

Neither  “total  life  cycle  system  management”  nor 
“total  life  cycle”  appears  in  SYS  201 B  Student  Guide. 
However,  in  SYS  201 B  Review  Resources  under 
Lesson  3:  Systems  Engineering,  the  “Total  Systems 
Approach”  states:  “The  PM  shall  be  the  single  point  of 
accountability  for  accomplishing  program  objectives 
for  total  life-cycle  systems  management,  including 
sustainment.  The  PM  shall  apply  human  systems 

•  Is  this  similar  to  Systems  Engineering  in  all  phases 
of  a  program — not  just  SDD?  There  are  lessons 
devoted  to  the  specific  acquisition  phases  (e.g., 
Concept  and  Technology  Development;  System 
Definition;  and  Design,  Fabrication  and  Test; 
Production:  IPPD  and  the  Paladin  Enterprise;  and 
Deployment,  Operation  and  Support),  and  each  has 
TLOs  that  include  how  systems  engineering  is  used 
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integration  to  optimize  total  system  performance 
(hardware,  software,  and  human),  operational 
effectiveness,  and  suitability,  survivability,  safety,  and 
affordability.  PMs  shall  consider  supportability,  life 
cycle  costs,  performance,  and  schedule  comparable 
in  making  program  decisions.  Planning  for  Operation 
and  Support  and  the  estimation  of  total  ownership 
costs  shall  begin  as  early  as  possible.  Supportability, 
a  key  component  of  performance,  shall  be  considered 
throughout  the  system  life  cycle.” 

within  that  phase 

In  service  systems  engineering 

This  hook  does  not  appear  to  have  been  explicitly 
called  out  in  any  of  the  SYS201  A  &  B  lesson 
materials. 

In  a  search  for  “in-service”  in  SYS  201  B:  Review 
Resources:  Under  System  Deployed  and  Post¬ 
production  in  Lesson  20:  Product  Improvement,  there 
is  one  section  that  specifically  touches  on  the  in- 
service  issue:  “Minor  modifications  are  less  costly 
than  major  modifications  and  therefore  are  handled  by 
each  service’s  In-Service  Engineering  Agent  (ISEA)  or 
the  program  office  if  it’s  a  cradle-to-grave  program. 

They  utilize  systems  engineering  activities  in  all  of  the 
life  cycle  phases.”  In  addition,  there  is  the  following 
information  on  the  operation  and  support  phase: 

•  Lesson  1:  Svstems  Acquisition  Overview  states 
that  RDT&E,  production,  and  operation  and  support 
objectives  have  to  all  be  considered  in  determining 
the  cost  of  a  program 

•  Under  Total  Svstems  Approach  in  Lesson  3: 

Systems  Engineering:  “Planning  for  Operation  and 
Support  and  the  estimation  of  total  ownership  costs 
shall  begin  as  early  as  possible.  Supportability,  a 
key  component  of  performance,  shall  be 
considered  throughout  the  system  life  cycle.” 

•  The  TLO  for  Lesson  13:  Production:  IPPD  and  the 
Paladin  Enterprise,  is  to  “evaluate  use  of  systems 
engineering  process  to  monitor  and  control  the 
system  configuration,  support  the  production 
process  and  control  the  program  cost  and  schedule.” 
The  TLO  shows  that  systems  engineering  is  being 
taught  in  conjunction  with  the  Production  phase.  A 
slide  on  p.  170  includes  “Functions  of  the  Systems 
Engineer  During  Production”  and  another  on  p.  176 
includes  “Test  and  Evaluation  &  Systems 

Engineering.”  [These  will  probably  be  incorporated 
into  the  new  201  course  that  is  by  phase.] 

•  Lesson  14:  Deployment,  Operation  &  Support  has  a 
TLO  that  incorporates  the  use  of  systems 
engineering  in  that  phase.  “Given  a  case  study 
students  will  be  able  to:  Evaluate  use  of  the  systems 
engineering  process  to  reduce  risk  of  operational/ 
support  problems,  as  well  as  to  plan  and  monitor  the 
fielding  process.” 

[These  next  comments  are  based  on  the  expanded 
definition  of  “in-service”  systems  engineering  from 
E10] 

•  Unable  to  find  a  reference  to  “in-service  systems 
engineering”  in  SYS  301  using  the  “find”  function. 
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•  A  question — who  actually  performs  “in-service” 
systems  engineering,  or  SE  during  deployment?  Is  it 
still  the  responsibility  of  the  system  developer 
(government  or  contractor?),  or  does  it  become  the 
responsibility  of  the  users?  Or  is  it  some  combination 
where  it  is  the  responsibility  of  the  developer  to  put 
in  place  a  mechanism  for  the  system  users  to  do  in- 
service  engineering?  This  hook  mentions  providing 
the  program  manager  with  information,  including 
“system  trends”  about  the  system.  Is  there  a  PM 
once  a  system  is  fielded?  And  are  the  “system 
trends”  maintenance  type  trends,  or  breakdown 
trends,  or  performance  trends  from  development?  Or 
trends  in  technologies  for  upgrades?  Or  trends  that 
help  determine  which  parts  of  the  system  may  wear 
more  than  others?  Etc. 

•  Should  something  about  “during  all  phases  of  the 
system  lifecycle”  be  added  to  the  TLO  for  Lesson  3: 
System  Analysis  and  Control?  The  TLO  currently 
reads:  “Apply  systems  analysis  and  control  tools, 
employing  an  Integrated  Product  and  Process 
Development  approach  to  systems  engineering 
management.”  Or  perhaps  it  could  be  included  in 
one  of  this  lesson’s  ELOs?  Another  place  to  include 
these  hooks  could  be  in  Lesson  14:  Deployment, 
Operation  and  Support.  Or,  potentially  Lesson  15: 
Improvements  to  Existing  Weapon  Systems. 

•  This  hook  fits  nicely  with  the  robust  SE  concept 

Monitor  experienced  svstem  performance  and  failures 

•  “Continue  to  monitor  system  performance”  is  listed 
as  an  “Operational  Support  Objective”  on  page  197 
of  Lesson  14:  Deployment,  Operation  &  Support 

•  Lesson  3:  System  Analysis  and  Control  includes 
Technical  Performance  Measures  (TPMs)  but  it  does 
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not  specifically  mention  using  them  during  the 
deployment/support  phase 

•  COSSI  (Commercial  Operations  &  Support  Savings 
Initiative)  is  described  in  two  slides  (p.  220)  in 

Lesson  15:  Improvements  to  Existing  Weapon 
Systems.  An  example  about  automatic  data 
collection  and  diagnostics  is  included. 

•  Lesson  6-03:  SW  Best  Practices  includes  “Life  Cycle 
Integration”  as  a  best  practice  on  p.  396  (this  was 
extracted  from  a  DSMC  presentation  called  PSM  or 
Practical  Software  Measurement).  The  idea  behind 
this  practice  is  that  measurement  results  must  be 
made  available  at  appropriate  times  in  the  software 
lifecycle  and  that  decisions  in  one  phase  may  have 
outcomes  in  other  lifecycle  phases. 

•  Lesson  6-03:  SW  Best  Practices  discusses  the  use 
of  metrics,  but  not  specifically  with  respect  to 
deployed  software.  However,  p.  469  does  include 
“Make  software  measurement  an  integral  part  of 
program  management  throughout  the  software  life 
cycle”  in  a  Metrics  Summary  slide. 

Identify  root  causes 

•  Using  the  find  function,  only  one  reference  to  the 
term  “root  cause”  occurs,  which  is  in  a  worksheet  in 
Lesson  10:  System  Definition — it  did  not  come  up  in 
Lesson  14:  Deployment,  Operations,  &  Support, 
Lesson  15:  Improvements  to  Existing  Weapon 
Systems,  or  Lesson  3:  System  Analysis  and  Control. 

Evaluate  risks 

•  Risk  identification,  evaluation,  handling  is  included  in 
Lesson  3,  but  again,  the  lesson  does  not  specifically 
mention  doing  this  in  the  deployment/support 
phases  of  a  system’s  lifecycle. 
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•  In  Lesson  15:  Improvements  to  Existing  Weapon 
Systems,  p.  226  includes  a  slide  suggesting  things 
that  a  SPRDE  Managers  need  to  do.  One  of  the 
statements  is  “Be  alert  for  changes  that  will  cause 
the  need  to  change  your  system.  Involve  the  User!” 
That  sounds  kind  of  like  identifying  risks? 

•  Lesson  6-03:  SW  Best  Practices  discusses  risk 
management,  but  not  specifically  with  respect  to 
deployed  software  (p.  462) 

Provide  the  PM  with  integrated  technical  assessment  of 
system  trends,  corrective  action  alternatives,  staffinq 

and  resource  requirements. 

•  Charts  showing  performance  measures  over  the 
lifecycle  of  a  systems’  development  (which  could  be 
considered  trends)  are  shown  in  Lesson  3  (p.  157 
has  an  example),  but  not  really  trends  in 
deployment. 

•  In  Lesson  15:  Improvements  to  Existing  Weapon 
Systems,  p.  226  includes  a  slide  suggesting  things 
that  a  SPRDE  Managers  need  to  do.  One  of  the 
statements  is  “Be  sure  to  look  carefully  at  operations 
and  support  activities  and  results  from  the  field  to  tell 
when  to  plan  for  a  system  change.”  Another  is  to 
“use  the  systems  engineering  process.” 

•  Lesson  6-01  includes  a  teaching  note,  Teaching 

Note  2,  “Post  Deployment  Software  Support”  (p. 
299-302)  that  gets  into  improving  the  PDSS  process. 
Developing  a  CRLCMP  or  Computer  Resources  Life 
Cycle  Management  Plan  is  one  of  the  actions 
suggested  to  improve  the  process.  It  “defines  criteria 
for  measuring  progress”  and  “identifies  the 
resources  needed  to  acquire,  develop,  test  and 
support  computer  resources  (e.g.,  facilities, 
personnel,  hardware,  software,  training,  funding, 
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tools).”  It  is  noted  that  this  plan  used  be  required  by 
DoD  policies  and  while  it  is  no  longer  required,  it  is 
still  a  best  practice. 

Metrics  (added) 

A  search  on  the  term  “metrics”  revealed  that  it  is  in  the 
SYS  201  B  Review  Resource  text  127  times.  The 
following  is  a  summary  of  where  and  how  it  appears: 

•  Metrics  first  appears  in  Lesson  2:  Integrated 

Product  and  Process  Development  under  the 
sections,  IPPD  Principles  and  Proactive 

Identification  of  Management  and  Risk,  where  it 
states:  “Technical  and  business  performance 
measurement  plans  should  be  developed,  with 
appropriate  metrics,  and  compared  to  best-in-class 
government  and  industry  benchmarks  to  verify  their 
effectiveness.”;  The  definition  of  metrics  is  also 
listed  as  one  of  nine  steps  to  implementing  IPPD 
for  a  program;  “Metrics  are  used  at  all  levels  of  a 
program’s  structure,  preferably  representing  key 
measures  of  output  rather  than  input  or  activity 
metrics.  Many  metrics,  such  as  those  relating  to 
cost,  schedule,  and  performance  (e.g.,  life  cycle 
costs)  can  be  used  throughout  the  program’s  life 
cycle,  while  others  may  be  tied  to  one  portion  of  the 
program.”;  Further  mention  of  metrics  also  appears 
in  the  following  sections:  IPPD  Metrics,  Metric 
Attributes,  Metric  Development  Process,  and  Metric 
Guidelines 

•  Under  the  section  on  Metrics  in  Management: 

Control  Criteria  in  Lesson  6:  Systems  Enqineerinq 
Planning  it  says:  “Management  of  technical 
activities  requires  three  basic  types  of  metrics: 
Product  metrics  that  track  the  development  of 
products,  Earned  value  that  tracks  conformance  to 
the  planned  schedule  and  cost,  and  Management 

•  Metrics  are  discussed  throughout  SYS  301  in  many 
of  the  lessons.  For  example,  the  TLO  for  Lesson  6- 
04:  Software  Management  Exercise  is  "Given  a 
software  intensive  system  scenario  and  project 
metrics  data,  conduct  an  in-depth  project 
performance  analysis  to  assess  the  project's  status, 
determine  problem  causes  and  recommend 
corrective  courses  of  action.”  One  of  the  TLOs  is  to 
“Evaluate  a  set  of  metrics  for  an  ongoing  program.” 

•  Metrics  come  up  in  Lesson  3:  System  Analysis  and 
Control  with  respect  to  RFPs  and  what  they  require. 

(p.  153)  They  should  have  key  TPMs  and  program 
metrics 

•  Overall,  “metrics”  come  up  most  often  in  the  four 
software  lessons 

•  A  teaching  note  in  Lesson  6-01 :  The  DoD  Software 
Acquisition  Management  Environment  mentions 
using  software  metrics  as  a  best  practice  (p.  311) 

•  Another  teaching  note  on  quality  in  Lesson  6-02  also 
talks  about  metrics  (p.  330).  It  says  that  metrics  and 
measurements  are  needed  in  order  to  know  if  a 
product  has  quality  attributes  (p.  330). 

•  Software  management  metrics  are  mentioned  later 
in  Lesson  6-02  as  a  way  to  examine  the  status  of  the 
software  at  various  intervals,  (p.  342) 

•  There  is  a  teaching  note  devoted  to  software  metrics 
on  p.  381  (lesson  6-03:  DoD  Software  Acquisition 
Planning  and  Strategy) 

•  In  Lesson  7:  Concept  and  Technology  Development, 
p.  608  includes  a  slide  that  says  that  showing  that 
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process  metrics  that  track  management  activities.”; 
Elsewhere  in  this  lesson,  product  metrics  are 
equated  with  systems  engineering  metrics  and 
describes  three  types  of  requirements  reflected  by 
product  metrics-  operational  performance,  lifecycle 
suitability,  and  affordability;  In  addition  it  also  states 
that  “A  key  set  of  systems  engineering  metrics  is 
TPM.” 

•  Metrics  appears  under  the  section  on  Software 
Management  Metrics  in  Lesson  10:  Verification 

•  Metrics  appears  under  the  section  on  Metrics  under 
Durinq  Operational  Support  in  Lesson  15:  Cost 
Containment 

•  In  the  section  on  Metrics  under  Risk  Management 
in  Lesson  17:  Risk  Management,  it  states:  “Earlv 
risk  planning  should  also  establish  risk  monitoring 
metrics.  These  metrics  allow  measurement  and 
evaluation  of  the  status  of  risk-handling  options, 
system  performance,  cost  and  schedule.” 

•  Under  the  section  on  Product  Metrics  in  Lesson  18: 
Technical  Performance  Measurement  it  states:  “A 
key  set  of  systems  engineering  metrics  are  the 

TPMs.  TPMs  are  product  metrics  that  track  design 
progress  toward  meeting  customer  requirements. 
They  are  closely  associated  with  the  SEP  because 
they  directly  support  traceability  of  operational 
needs  in  the  design  effort.”  Under  the  section  on 
Timing  it  further  states:  “Product  metrics  which 
track  the  development  of  the  product  are  tied 
directly  to  the  design  process.  Planning  for  metrics 
should  be  part  of  the  Systems  Engineering  Process 
(SEP)  planning  effort  starting  in  the  early  phases  of 
the  life  cycle.”  Furthermore,  “The  need  to  track 
product  metrics  ends  in  the  production  phase, 

“metrics  for  future  successful  phases  [are]  defined” 
is  a  purpose  of  a  Concept  Exploration  Work  Effort 
Technical  Review. 

•  Lesson  1 1 :  Design,  Fabrication  and  Test  uses  the 
term  “metrics”  in  a  couple  of  places:  first  to  with 
respect  to  preliminary  design  (p.  132)  and  then 
again  on  a  slide  about  Production  Readiness 

Reviews  (p.  139) 

•  “Building  a  Business  Case  for  Modeling  and 
Simulation,”  an  article  included  in  Lesson  16: 

Modeling  and  Simulation  mentions  creating  metrics 
as  the  final  step  in  developing  a  business  case. 
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usually  concurrent  with  the  establishment  of  the 
product  (as-built)  baseline.” 

Technical  Basis  for  Cost 

•  Specific  references  to  the  “technical  basis  for  cost” 
and  “cost  drivers”  do  not  appear  in  SYS  201  B: 
Review  Resource.  However,  the  focus  on  CAIV 
and  Total  Ownership  Cost  in  Lesson  15:  Cost 
Containment  does  appear  to  capture  the  intent  of 
this  hook.  Cost  objectives  are  to  be  set  in  an 
aggressive,  yet  realistic,  manner  through  cost- 
performance  analyses  and  trades  based  upon 
balancing  “mission  needs  with  projected  out-year 
resources,  taking  into  account  existing  technology, 
maturation  of  new  technologies  and  anticipated 
process  improvement  in  both  DoD  and  industry.” 
Furthermore,  these  cost  objectives  are  to  be 
established  early  and  updated  for  each  acquisition 
phase. 

•  A  teaching  note  on  p.  1 37  of  Lesson  3:System 
Analysis  and  Control  Tools,  discusses  the  need  to 
use  Technical  Performance  Measures  (TPMs)  to 
provide  visibility  into  “key  elements  of  the  work 
breakdown  structure,  especially  those  which  are 
cost  drivers  on  the  program,  lie  on  the  critical  path, 
or  which  represent  high  risk  items.” 

•  CAIV  is  included  in  Lesson  3:  System  Analysis  and 
Control  Tools. 

•  A  slide  on  p.  1 62  suggests  that  a  SPRDE  manager 
should  use  Earned  Value  as  an  indicator  of  cost  and 
risk  drivers 

•  Slides  on  pages  470  and  471  of  Lesson  6-03: 

Software  Best  Practices  include  material  about  cost 
drivers  and  cost  estimating  for  software. 

•  Environmental,  Safety  and  Occupational  Health 
(ESOH)  cost  drivers  are  listed  on  pages  17  and  18 
of  Lesson  8:  ESOH 

Requirements  in  an  Integrated 
Framework 

•  Specific  references  to  an  “integrated  framework” 
do  not  appear  in  SYS  201 B.  However,  with 
coverage  of  IPPD,  IPTs,  and  the  total  systems 
approach,  one  may  infer  that  this  hook  is  touched 
upon. 

•  “Identify  and  describe  the  major  ESOH 
requirements”  is  one  of  the  ELOs  for  Lesson  8: 

ESOH  (p.  7) 

•  There  is  mention  of  both  manufacturing  and  support 
requirements  in  Lesson  11:  Design,  Fabrication  and 
Test,  p.  132. 

•  One  of  the  Logistics  objectives  listed  on  p.  194  of 
Lesson  14:  Deployment,  Operation  and  Support,  is 
to  “Develop  and  acquire  a  system  that  cost 
effectively  meets  the  user  readiness  and  support 
requirements.”  This  lesson’s  overview  states  that 
“[The  student]  needs  a  holistic  understanding  of  how 
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requirements  fit  into  the  context  of  the  overall 
systems  engineering  process,  especially  those 
factors  relating  to  equipment  life  cycle  in  the  DoD 
environment.” 

•  There  is  also  a  lesson  learned  on  p.  202  of  Lesson 

14  that  states  that  “Logistics  needs  are  systems 
engineering  process  requirements  -  not  constraints” 

•  The  TLO  for  Lesson  16:  Modeling  and  Simulation  is 
“Identify  the  modeling  &  simulation  requirements, 
benefits,  pitfalls,  planning  and  applications  in 
systems  acquisition.” 

•  There  is  a  slide  on  page  171  in  Lesson  3:  System 
Analysis  and  Control  Tools  titled  “Requirements 
Flowdown”  that  covers  the  need  to  relate  operational 
requirements  from  the  CDD  to  system  specifications. 

•  “Lack  of  requirements  traceability,  ORD  — >  Spec”: 
The  lack  of  requirements  traceability  from  the  ORD 
to  specs  is  one  of  the  reasons  listed  for  why  systems 
pass  DT&E  but  fail  IOT&E  on  p.  142  of  Lesson  1 1 : 
Design,  Fabrication  and  Test. 

Appendix  F 

Systems  Engineering  Topics 


SYSTEMS  ENGINEERING  TOPICS 


IDA  created  this  list  of  topics  from  various  standards;  models;  textbooks;  and  education,  training,  and  certification 
programs  as  a  tool  to  support  our  development  of  the  SPRDE/SE  Duties  and  Tasks  list.  We  developed  the  tables  intending  to 
review  the  list  of  Duties  and  Tasks  against  them  and  make  sure  that  we  did  not  miss  any  important  systems  engineering  topics 
in  the  list.  This  appendix  includes  six  tables  of  systems  engineering  topics. 

•  Standards  and  Models 

•  Systems  Engineering  Textbooks 

•  Education  and  Training  Programs 

•  Certification  Programs  (1) 

•  Certification  Programs  (2) 

•  Certification  Programs  (3) 

All  six  of  the  tables  are  organized  according  to  a  general  list  of  systems  engineering  topics  located  in  the  first  column 
of  each  table.  There  are  typically  five  or  six  sources  per  table.  The  topics  from  each  source  are  below  the  title  and  are 
organized  according  to  the  general  list  of  topics  in  the  left  column.  This  organization  provided  a  structure  to  the  numerous 
topics  from  each  source.  As  a  result,  however,  there  are  many  empty  cells  in  all  of  the  tables.  Note  that  some  of  the  cell  entries 
are  followed  by  an  underscore  (_)  and  a  number  and  are  highlighted  in  blue.  We  used  this  representation  to  show  that  the 
contents  of  that  cell  are  mapped  to  other  general  topics  in  the  table.  For  example,  “Requirements  Validation”  from  Source  #1 
may  be  mapped  to  both  “Requirements  Development”  and  to  “Validation”  in  the  left-hand  column  of  the  table.  In  this  case, 
“Requirements  Validation  l”  would  appear  in  the  Source  #1  Requirements  Development  cell,  and  “Requirements 
Validation_2”  would  appear  in  the  Source  1  Validation  cell. 


SYSTEMS  ENGINEERING  TOPICS:  STANDARDS  AND  MODELS 


Sources  for  this  table: 

•  INCOSE  SE  Book  of  Knowledge  (SEBOK),  “Mapping  SE  Processes  into  Program  Phases,”  INCOSE 
Handbook,  p.  22,  available  on  SEBOK,  available  at  http://proiects.metrostarsystems.com:8080/incose/pal/ 

•  INCOSE  SEBOK,  SE  Competencies,  2.3.3  SE  Competencies,  available  at 
http://proiects.metrostarsystems.com:8080/incose/pal/ 

•  Capability  Maturity  Model  Integration  (CMMI),  Continuous  version  updated  2002,  available  at 
http://www.sei.cmu.edu/pub/documents/02.reports/pdf/02tr01 1.pdf 

•  ISO/EIC  15288,  first  edition  2002-1 1-01,  International  Standard,  SE — System  life  cycle  processes 

•  ANSI/EIA-632-1998,  Approved  January  7,  1999,  International  Standard,  SE — System  life  cycle  processes 

•  IEEE  Std  1220 — 1998,  International  Standard,  SE — System  life  cycle  processes 


Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

SE  Process 

SE  Process 
(SEP) 

Policies  and 
procedures  for  SE 

Req’ts 

Development 

Business 

Processes  and 
Operational 
Assessment 
(BPOA) 

Engineering: 

Req’ts 

Development 

Technical 
Processes:  Req’ts 
Analysis  Process 

Req’ts  Definition 
Process  Req’ts: 
Acquirer  Req’ts 

SEP:  Req’ts 
Analysis 

Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Req’ts  Analysis: 
Capture  Source 
Req’ts 

Business 

Processes  and 
Operational 
Assessment 
(BPOA):  Business 
&  operational 
shortfalls 

Technical 

Processes: 

Stakeholder 

Req’ts  Definition 
Process 

Req’ts  Definition 
Process  Req’ts: 
Other 

Stakeholder 

Req’ts 

SEP:  Req’ts 
Verification^ 

Req’ts  Analysis: 
Develop 
Operational 
Concept 

Business 

Processes  and 
Operational 
Assessment 
(BPOA):  Solution 
Req’ts 

Req’ts  Definition 
Process  Req’ts: 
System  Technical 
Req’ts 

Req’ts  Analysis: 

Functional 

Performance 

Req’ts 

Solution  Definition 
Process  Req’ts: 
Specified  Req’ts 

Req’ts  Analysis: 
Design  Constraint 
Req’ts 

Req’ts  Analysis: 
Req’ts  Allocation 

Req’ts  Mgmt 

Engineering: 

Req’ts  Mgmt 

Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Functional 

Analysis  and 
Allocation 

System  /  Solution 
/  Test 

Architecture 
(SSTA):  Develop 
functional 
architecture 

Solution  Definition 
Process  Req’ts: 
Logical  Solution 
Representations 

SEP:  Functional 
Analysis 

System  /  Solution 
/  Test 

Architecture 
(SSTA):  Allocate 
functions  to 
physical 
elements_1 

SEP:  Functional 
Verification^ 

Physical  Solution/ 
Design  Synthesis 

System  /  Solution 
/  Test 

Architecture 
(SSTA):  Develop 
physical 
architecture 

Engineering: 

Technical 

Solution 

Technical 

Processes: 

Implementation 

Process 

Solution  Definition 
Process  Req’ts: 
Physical 

Solutions 

Representations 

SEP:  Synthesis 

System 

Architecture 

Synthesis: 

Synthesize 

Multiple 

Architectures 

System  /  Solution 
/  Test 

Architecture 
(SSTA):  Allocate 
functions  to 
physical 
elements_2 

Technical 

Processes: 

Architectural 

Design  Process 

Implementation 
Process  Req’ts: 
Implementation 

Development 

Strategies 

Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

System 

Architecture 

Synthesis: 

System  Element 
Req’ts 

System 

Implementation 

Support 

System 

Verification 

Process  Req’ts: 
Design  Solution 
Verification^ 

System 

Architecture 
Synthesis: 
Eval/Select  Pfd. 
Architecture 

System 

Architecture 
Synthesis: 
Integrated  Sys. 
Physical  Config. 

System 

Architecture 

Synthesis: 

Define/Refine 

lnterfaces_1 

System 

Architecture 

Synthesis: 

Develop  Spec. 
Trees  &  Specs. 

Systems  Analysis 

Systems  Analysis 

Modeling, 
Simulation,  & 
Analysis  (MS&A) 

Support: 

Measurement  and 
Analysis 

Systems  Analysis 
Process  Req’ts: 
Effectiveness 
Analysis 

SEP:  Systems 
Analysis 

Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

MS&A:  Conduct 
system 
performance 
modeling  and 
forecasting 

Support:  Causal 
Analysis  and 
Resolution 

SEP:  Control 

MS&A:  Conduct 
system 
architecture 
modeling  and 
analysis 

Systems 

Analysis:  Tradeoff 
studies 

Systems  Analysis 
Process  Req’ts: 
Tradeoff  Analysis 

Systems 

Analysis:  System 
Modeling  & 
Simulation 

Modeling  and 
Prototyping^ 

Integration 

System 

Implementation 
Support:  System 
Integration 

Engineering: 

Product 

Integration 

Technical 

Processes: 

Integration 

Process 

Support: 
Organizational 
Environment  for 
Integration^ 

Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Verification 

System 

Implementation 
Support:  System 
Verification 

Engineering: 

Verification 

Technical 

Processes: 

Verification 

Process 

System 

Verification 

Process  Req’ts: 
Design  Solution 
Verification_2 

SEP:  Req’ts 
Verification_2 

System 

Verification 

Process  Req’ts: 

End  Product 
Verification 

SEP:  Functional 
Verification_2 

System 

Verification 

Process  Req’ts: 
Enabling  Product 
Readiness 

SEP:  Design 
Verification 

Validation 

Engineering: 

Validation 

Technical 

Processes: 

Validation 

Process 

Req’ts  Validation 
Process  Req’ts: 
Requirement 
Statements 
Validation 

Req’ts  Validation 
Process  Req’ts: 
Acquirer  Req’ts 
Validation 

Req’ts  Validation 
Process  Req’ts: 
Other 

Stakeholder 

Req’ts  Validation 

Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Req’ts  Validation 
Process  Req’ts: 
Logical  Solution 
Representations 
Validation 

End  Products 
Validation 

Process  Req’ts: 

End  Products 
Validation 

SE  Mqmt 

Project  Mgmt: 
Quantitative 

Project  Mgmt 

Enterprise 

Processes: 

System  Life  Cycle 
Processes  Mgmt 
Process 

Enterprise 
Processes: 
Resource  Mgmt 
Process 

SE  Process 
Environment 

Support: 
Organizational 
Environment  for 
lntegration_2 

Enterprise 

Processes: 

Enterprise 

Environment 

Mgmt  Process 

Planning  Process 
Req’ts:  Schedule 
and 

Organization^ 

SE  Planning  and 
Strategy 

Mgmt:  Risk, 
Configuration, 
Baseline  (Mgt): 
Tailor  SE  process 

Project  Mgmt: 
Project  Planning 

Project 

Processes: 

Project  Planning 
Process 

Planning  Process 
Req’ts:  Process 
Implementation 
Strategy 

Planning  the 
technical  effort: 
Master  Schedule 

Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Support:  Decision 
Analysis  and 
Resolution 

Project 

Processes: 

Decision-making 

Process 

Planning  Process 
Req’ts:  Technical 
Effort  Definition 

Planning  the 
technical  effort: 
Detail  Schedule 

Planning  Process 
Req’ts:  Schedule 
and 

Organ  ization_2 

Planning  the 
technical  effort: 
Technical  Plans 

Planning  Process 
Req’ts:  Work 
Directives 

System 

Breakdown 

Structure 

SE  Process 

Control:  SEMP, 
SEMS/SEDS, 

TPM,  Audits_1 

Planning  Process 
Req’ts:  Technical 
Plans 

Planning  the 
technical  effort: 
Engineering  Plan 

Risk  Mgmt 

System  Analysis: 
Risk  Mgmt 

Mgmt:  Risk, 
Configuration, 
Baseline  (Mgt): 
Conduct  risk 

Mgmt 

Project  Mgmt: 

Risk  Mgmt 

Project 

Processes:  Risk 
Mgmt  Process 

Systems  Analysis 
Process  Req’ts: 
Risk  Analysis 

Mgmt:  Risk, 
Configuration, 
Baseline  (Mgt): 
Manage  key 
drivers 
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Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Configuration 

Mgmt 

SE  Product 

Control 

Mgmt:  Risk, 
Configuration, 
Baseline  (Mgt): 
Perform  system 
configuration 

Mgmt 

Support: 

Configuration 

Mgmt 

Project 

Processes: 

Configuration 

Mgmt  Process 

System 

Implementation 
Support:  Baseline 
Maintenance 

Data  Mgmt 

Project 

Processes: 
Information  Mgmt 
Process 

Control  Process 
Req’ts: 

Information 

Dissemination 

Integrated 
Database:  Data 
and  Schema 

Integrated 
Database:  Tools 

Integrated  Data 
Package: 

Hardware 

Integrated  Data 
Package: 

Software 

Integrated  Data 
Package: 

Lifecycle 

Processes 

Integrated  Data 
Package:  Human 
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Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Interface  Mgmt 

System 

Architecture 

Synthesis: 

Define/Refine 

lnterfaces_2 

System  /  Solution 
/  Test 

Architecture 
(SSTA):  Develop 
interface 
architecture 

MS&A:  Conduct 
system  user 
interface 
analysis_1 

Manufacturing 
Readiness  Levels 

System  /  Solution 
/  Test 

Architecture 
(SSTA):  Develop 
the  manufacturing 
system 
architecture 

Cost  Analysis 

System  Analysis: 
Life  Cycle  Cost 
Analysis 

Life  Cycle  Cost  & 
Cost  Benefit 
Analysis  (LCC  & 
CBA) 

Life  Cycle  Cost  & 
Cost  Benefit 
Analysis  (LCC  & 
CBA):  Perform 
activity-based 
costing 
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Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Life  Cycle  Cost  & 
Cost  Benefit 
Analysis  (LCC  & 
CBA):  Conduct 
life  cycle  costing 

Life  Cycle  Cost  & 
Cost  Benefit 
Analysis  (LCC  & 
CBA):  Conduct 
cost  benefit 
analyses  of  the 
system 
engineering 
process 

Life  Cycle  Cost  & 
Cost  Benefit 
Analysis  (LCC  & 
CBA): 

Understand  the 
system  cost 
drivers 

Life  Cycle  Cost  & 
Cost  Benefit 
Analysis  (LCC  & 
CBA):  Estimate 
total  ownership 
cost 
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Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

System  Analysis: 
Cost  and 
Effectiveness 
Analysis 

DoD  Acquisition 

Related  Topics 

Agreement 

Processes: 

Acquisition 

Process 

Acquisition 

Process  Req'ts: 
Product 

Acquisition 

Agreement 

Processes: 

Supply  Process 

Supply  Process: 
Product  Supply 

Technical  Inputs 
to  RFP 

Pre-proposal 
activities:  Mission, 
SRD,  SOW,  RFP, 
CDRL 

Acquisition 

Process  Req'ts: 

Supplier 

Performance 

Project  Mgmt: 
Supplier 

Agreement  Mgmt 

Project  Mgmt: 
Integrated 

Supplier  Mgmt 
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Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Technical 

Reviews  amd 
Audits 

SE  Process 

Control:  SEMP, 
SEMS/SEDS, 

TPM,  Audits_2 

Project  Mgmt: 
Project  Monitoring 
and  Control 

Project 

Processes: 

Project 

Assessment 

Process 

Assessment 
Process  Req’ts: 
Progress  Against 
Plans  and 
Schedules 

Technical 

Reviews 

Project 

Processes: 

Project  Control 
Process 

Assessment 
Process  Req’ts: 
Progress  Against 
Req’ts 

Assessment 
Process  Req’ts: 
Technical 

Reviews 

SE  Enablers 

IPPD 

Project  Mgmt: 
Integrated 

Teaming 

Integration  of  the 

SE  effort: 

Integrated  Teams 

Project  Mgmt: 
Integrated  Project 
Mgmt  for  IPPD 

Product  and 
Process 

Improvement:  Re¬ 
engineering 

Product  and 

Process 

Improvement: 

Self  Assessment 
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Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Product  and 
Process 
Improvement: 
Lessons  Learned 

Prototyping 

Modeling  and 
Prototyping_2 

SE  Desiqn 
Considerations 

Training 

System  /  Solution 
/  Test 

Architecture 
(SSTA):  Develop 
the  training 
system 
architecture 

Reliability, 

Availability, 

Maintainability 

Technical 

Processes: 

Maintenance 

Process 

Test  and 

Evaluation 

System  /  Solution 
/  Test 

Architecture 
(SSTA):  Develop 
test  system 
architecture 
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Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Sustainability 

System 

Implementation 

Support: 

Sustaining 

Engineering 

Disposal  and 

Demilitarization 

Considerations 

Technical 

Processes: 

Disposal  Process 

Quality 

Support:  Process 
and  Product 

Quality 

Assurance 

Enterprise 

Processes: 

Quality  Mgmt 
Process 

Quality  Mgmt 

Supportability/ 

Acquisition 

Logistics 

Technical 

Processes: 

Operation 

Process 

Serviceability  / 
Logistics  (S/L): 
Develop  system 
supply  support 
and  spares  Mgmt 

Serviceability  / 
Logistics  (S/L): 
Analyze  system 
operational  and 
servicing  skill 

Req’ts 
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Source  Title 

INCOSE  SEBOK 

“Mapping  SE 
Processes  into 
Program  Phases” 

INCOSE  SEBOK 

SE  Competencies 

CMMI 

ISO/EIC  15288, 
first  edition 
2002-11-01 

ANSI/EIA-632- 

1998 

IEEE  Std  1220- 
1998 

Serviceability  / 
Logistics  (S/L): 
Develop  system 
and  platform 
documentation 

Human  Systems 
Integration 

MS&A:  Conduct 
system  user 
interface 
analysis_2 

Other 

Enterprise 
Processes: 
Investment  Mgmt 
Process 

Transition  to  Use 
Process  Req’ts: 
Transition  to  Use 

Integration  of  the 

SE  effort: 

Concurrent 

Engineering 

Control  Process 
Req’ts:  Outcomes 
Mgmt 

Specification  Tree 

Drawing  Tree 
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SE  TOPICS:  SE  TEXTBOOKS 


Sources  for  this  table: 

•  Engineering  Design  Process,  Priest 

•  Systems  Engineering  and  Analysis,  Blanchard  and  Fabrycky 

•  Engineering  Complex  Systems  with  Models  and  Objects,  David  Oliver,  Timothy  P.  Kelliher,  and  James  G. 
Keegan,  Jr. 

•  The  Engineering  Design  of  Systems,  Dennis  Buede 

•  Introduction  to  SE,  Sage  and  Armstrong 


Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

SE  Process 

Core  Technical 

Process:  Process, 
Methodology  and  Tools 

Core  Technical 

Process:  Product  Life 
Cycle,  Acquisition,  SE 
Process 

Core  Technical 

Process:  The  SE 

Process  Model 

Core  Technical 

Process:  Hierarchy, 
Waterfall,  Top  Down 
Development 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

Core  Technical 

Process:  Union  of  Best 
Practice  with  Modeling 

Req’ts  Development 

Req’ts  Definition: 
Market  Research 
and  Analysis 

Conceptual  System 
Design:  Identification 
of  a  Need 

Assess  Available 
Information:  A  Req’ts 
Taxonomy 

Develop  operational 
concept 

Req’ts  and 
Specifications 

Req’ts  Definition: 
Customer  Req’ts  and 
needs 

Conceptual  System 
Design:  System 

Req’ts  Analysis 

Assess  Available 
Information:  A 
"Behavior"  for  Assess 
Available  Information 
[this  refers  to  the 
process  used  to  Assess 
Available  Information.  It 
includes:  gathering 
heritage  information, 
gathering  user 
information,  gathering 
text  Req’ts  information, 
gathering  Ops  Concept 
Information,  etc). 

Define  system 
boundary  with 
external  systems 
diagram 

Formulation: 

Problem  Definition 

Req’ts  Definition: 
System  Req'ts, 
including  producibility 
and  reliability  l 

Conceptual  System 
Design:  System 
Specification 

Define  Effectiveness 
Measures 

Develop  system 
objectives  hierarchy 

Formulation:  Value 
System  Design 

Conceptual  Design: 

System 

specifications 

Preliminary  System 
Design:  Req’ts 
Allocation_1 

Develop,  analyze 
and  refine  Req’ts 
(originating  and 
system) 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

Conceptual  Design: 
Design  Req’ts 

Preliminary  System 
Design:  Design 

Req’ts  (Parameters) 

Ensure  Req’ts 
feasibility 

Conceptual  Design: 
Design  guidelines 

System  Test  and 
Evaluation:  Req’ts  for 
Test  and 

Evaluation_1 

Define  the  test 
system  Req’ts_1 

Conceptual  Design: 
Design  to  cost_1 

Detailed  Design: 
Detailed  design 
specifications^ 

Req’ts  Mgmt 

Preliminary  System 
Design:  Req’ts 
Allocation_2 

Req’ts  Mgmt 

Functional  Analysis 
and  Allocation 

Conceptual  Design: 
Functional  Allocation 

Conceptual  System 
Design:  Functional 
Analysis  and 

Allocation 

Basics  of  Behavior 
[refers  to  behavior 
modeling,  E.g., 

Functional  Flow  Block 
Diagrams,  Data  Flow 
Diagrams, 

Representation  of 
Behavior  as  State, 
Information  Model  for 
Behavior,  Relationship 
of  Behavior  and 

Structure] 

Functional 

Architecture 

Development 

Functional 
Decomposition  and 
Functional  Analysis 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

Preliminary  System 
Design:  Req’ts 
Allocation_3 

Create  a  Behavior 

Model 

Functional  Analysis 
and  Business 

Process 

Reengineering 

Preliminary  System 
Design:  Subsystem 
Functional  Analysis 

Physical  Solution/ 
Design  Synthesis 

Detailed  Design: 
Detailed  design 
specifications_2 

Conceptual  System 
Design:  Synthesis, 
Analysis,  and 
Evaluation_1 

Basics  of  Structure 
[E.g.,  Objects  and 
Classes,  Aggregation, 
Cardinality, 
Interconnection  of 
Objects,  Allocation  of 
Functions  to  Objects] 

Physical  Architecture 
Development 

Prelimary 

Conceptual  Design 
and  System 
Architecting 

Detailed  Design: 

Circuit  design,  parts 
selection,  component 
design,  part 
qualification, 
mechanical  design, 
thermal  design 
(combination  of 
many  topics) 

Preliminary  System 
Design:  Synthesis 
and  Design  Definition 

Create  a  Structure 

Model 

Operational 

Architecture 

Development 

Formulation: 

System  Synthesis 

Detailed  Design: 
Package  design 

Detail  Design  and 
Development: 

Design  Engineering 
Activities 

Interface  Design_2 

Detailed  Design, 
Production,  and 
Testing_1 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

Detailed  Design: 
Design  to  cost 

Detail  Design  and 
Development: 

System  Prototype 
Development 

Logical  Design  and 
Product  Architectural 
Specifications 

Detailed  Design: 
Documentation 

Systems  Analysis 

Detailed  Design: 
Analysis,  modeling, 
simulation,  and 
prototypes_1 

Conceptual  System 
Design:  Synthesis, 
Analysis,  and 
Evaluation_2 

Concept  Analysis 

Graphical  Modeling 
Techniques 

Analysis: 

Refinement  of  the 
Alternatives 

Detailed  Design: 

Make  or  buy  analysis 

Conceptual  System 
Design:  Conceptual 
Design  Review_1 

System  Analysis 

Quality  Function 
Deployment 

Optimization  in 

Design  and 

Operations 

Subsystem  Analysis 

Feasibility  Studies 

Control  Concepts 
and  Techniques 

Analysis  of  Systems 
with  Uncertain  and 
Imperfect  Information 

Structural  Modeling: 
Trees,  Causal  Loops, 
Influence  Diagrams 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

System  Dynamics 
Models  and 

Extensions 

Discrete  Event 

Models 

Tradeoff  Studies 

Conceptual  Design: 
Trade  studies 

Perform  Trade-off 
Analyses 

Decision  Analysis  for 
Design  Trades 

M&S 

Conceptual  Design: 
Simulation  and 
modeling 

Analysis:  System 
Analysis  and 

Modeling 

Detailed  Design: 
Analysis,  modeling, 
simulation,  and 
prototypes_2 

Technical 

Performance 

Measures 

Conceptual  System 
Design:  Technical 
Performance 

Measures  (TPM) 

Integration 

Detail  Design  and 
Development: 
Integrating  System 
Elements 

Integration  and 
Qualification 

SE  Mqmt 

Interface  with 

Acquisition  and  Mgmt_1 

SE  Methods  for  SE 
Mgmt 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

Pragmatics  of 

Systems  Mgmt 

SE  Process 
Environment 

SE  Planning  and 
Organization: 
Organization  for  SE 

SE  Organizational 
Structures 

Program  Mgmt  and 
Control:  Organization 
Goals  and  Objectives 

SE  Planning  and 
Strategy 

Conceptual  Design: 
Program  plans 

Conceptual  System 
Design: 

Accomplishment  of 
Feasibility  Analysis 

Create  Build  and  Test 
Plan_1 

Human  and 

Cognitive  Factors  in 

SE  and  Systems 
Mgmt_1 

Conceptual  System 
Design:  Advanced 
System  Planning 

Interpretation: 

Decision  Making 

Alternatives  and 
Models  in  Decision 
Making 

Interpretation: 

Planning  for  Action 

SE  Planning  and 
Organization:  SE 
Planning 

Formal  Decisions 

Program  Mgmt  and 
Control:  Direction 
and  Control  of 

Program  Activities 

Group  Decision 

Making  and  Voting 

F-25 


Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

SEMP  (SE  "Master" 
Plan) 

SE  Planning  and 
Organization:  SE 

Mgmt  Plan  (SEMP) 

Create  Build  and  Test 
Plan_2 

Risk  Mgmt 

Program  Mgmt  and 
Control:  Program 

Risk  Mgmt 

Interface  Mgmt 

Interface  Design_2 

Cost  Analysis 

Conceptual  Design: 
Design  to  cost_2 

Models  for  Economic 
Evaluation 

Economic  Models 
and  Economic 

System  Analysis 

Lifecycle  Cost 

Analysis 

Design  for 

Affordability: 
Introduction  to  Life 
Cycle  Costing 

Design  for 

Affordability:  Cost 
Emphasis  in  the 
System  Life  Cycle 

Design  for 

Affordability:  The  Life 
Cycle  Cost  Analysis 
Process 

DoD  Acquisition 

Related  Topics 

Interface  with 

Acquisition  and  Mgmt_2 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

Technology  Insertion 
Mgmt 

Detailed  Design:  Off¬ 
line  maturing  of  new 
technologies 

Preliminary  System 
Design:  Engineering 
Design  Technologies 

Technical  Reviews 
and  Audits 

Preliminary  System 
Design:  System 

Design  Reviews 

Detail  Design  and 
Development:  Detail 
Design  Reviews 

Conceptual  System 
Design:  Conceptual 
Design  Review_2 

SE  Enablers 

Software  Design  and 
Development 

Detailed  Design: 
Software  design 

Prototyping 

Detailed  Design: 
Analysis,  modeling, 
simulation,  and 
prototypes_3 

SE  Desiqn 
Considerations 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

Reliability, 

Availability, 

Maintainability 

Req’ts  Definition: 
System  Req’ts, 
including  producibility 
and  reliability_2 

Design  for  Reliability 

Operational 
Functioning  and 
Maintenance 

Design  for 
Maintainability 

Reliability, 

Availability, 
Maintainability,  and 
Supportability 
Models_1 

Test  and  Evaluation 

Detailed  Design: 
Testability 

System  Test  and 
Evaluation:  Req’ts  for 
Test  and 

Evaluation_2 

Define  the  test 
system  Req’ts_2 

Detailed  Design, 
Production,  and 
Testing_2 

Detailed  Design: 

Test  planning 

System  Test  and 
Evaluation: 

Categories  of 

System/  Component 
Testing 

Operational  Test  and 
Evaluation 

Test  and  Evaluation 
(with  subtopics) 

System  Test  and 
Evaluation:  Planning 
for  Test  and 

Evaluation 

System  Test  and 
Evaluation: 

Preparing  for  Test 
and  Evaluation 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

System  Test  and 
Evaluation:  Test 
Performance  and 
Reporting 

Design  for 

Producibility  and 
Disposabilityl 

Producability  & 
Manufacturability 

Detailed  Design: 

Production 

engineering 

Detailed  Design, 
Production,  and 
Testing_3 

Detailed  Design: 

Manufacturing 

planning 

Detailed  Design: 
Producibility 

Sustainability 

Production  and 
Sustaining 

Engineering  (with 
subtopics) 

Disposal  and 

Demilitarization 

Considerations 

Design  for 

Producibility  and 
Disposability_2 

Quality 

Detailed  Design: 
Quality  engineering 
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Source  Title 

Engineering  Design 
Process,  Priest 

Systems 
Engineering  and 
Analysis,  Blanchard 
and  Fabrycky 

Engineering  Complex 
Systems  with  Models 
and  Objects,  David 
Oliver,  et.  al. 

The  Engineering 
Design  of  Systems, 

Buede 

Introduction  to  SE, 

Sage  and  Armstrong 

Detailed  Design: 
Quality  specifications 

Supportability/ 
Acquisition  Logistics 

Detailed  Design: 
Logistics  engineering 

Design  for 
Supportability 

Operational 

Implementation 

Reliability, 

Availability, 
Maintainability,  and 
Supportability 
Models_2 

ESOH 

Detailed  Design: 

Safety  engineering 

Detailed  Design: 

Environmental 

testing 

Human  Systems 
Integration 

Detailed  Design: 
Human  engineering 

Design  for  Usability 
(Human  Factors) 

Human  and 

Cognitive  Factors  in 

SE  and  Systems 
Mgmt_2 

Other 

Detail  Design  and 
Development:  Detail 
Design  Aids 

Hand-off  (refers  to 
transition  between 
system  design  and 
design  work  b  individual 
engineering 
disciplines/suppliers) 

Obtain  approval  of 
system 

documentation 
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SE  TOPICS:  EDUCATION  AND  TRAINING  PROGRAMS 


Sources  for  this  table: 

•  DAU  SPRDE  SYS  201 A 

•  DAU  SPRDE  SYS  20  IB 

•  DAU  ASPRDEC  SYS  301 

•  Boeing  Commercial  Sources  Database 

•  The  National  Cryptologic  School’s  SE  Training  Program,  Program  Recommendations,  Final  Release,  12 
February  2003 

•  IBM  Req’ts,  For  purposes  of  mapping  to  Academic  Courses 


Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

SE  Process 

Systems 

Engineering 

SE  Process 

Outputs 

Introduction  to  SE 

SE  and 

Architecture 
Overview:  Define 
SE  and 

Architecture 

SE  Process 

Outputs 

Introduction  to 

SE:  working 
definition  of  SE 

SE  and 

Architecture 
Overview:  Define 
the  value  of  SEA 
to  a  project 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Introduction  to 

SE:  design 

SE  and 

Architecture 
Overview: 
Summarize  the 
key  elements  of 
the  SEA 
methodology 

SE  and 

Architecture 
Overview:  Clarify 
and  explain 
relationship 
between  the  SEA 
methodology  and 
other  processes, 
including  BTOP 

SE  and 

Architecture 
Overview:  Clarify 
and  explain 
relationship 
between  the  SEA 
methodology  and 
development 
models  such  as 
the  waterfall  and 
spiral  models_1 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

SE  and 

Architecture 
Overview:  Explain 
the  SEA  roles  and 
responsibilities 
and  integration 
with  other 
organizations 
throughout  the 
project  life  cycle 

SE  and 

Architecture 
Overview:  Instruct 
the  SEA  work 
products 
(deliverables) 

System 

Engineering 

Architecture 

Method:  Instruct 
the  complex 
problem  solving 
process;  concept 
of  system  design, 
analysis,  and 
evaluation 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Req’ts 

Development 

Req’ts  Analysis 

Req’ts  Analysis 

Business 

Processes  and 

Operational 

Assessment: 

Business  & 

operational 

shortfalls 

Introduction  to 

SE:  conceptual 
development 

SE  Architecture 
Req’ts: 

Distinguish 
between  Req’ts 
(whats)  and 
designs  (hows) 

Business 

Processes  and 
Operational 
Assessment: 
Solution  Req’ts 

Req’ts:  translating 
customer  needs 
and  priorities  into 
an  operational 
concept  and  then 
into  traceable 
functional  and 
system 
performance 

Req’ts 

SE  Architecture 
Req’ts:  Define  the 
characteristics  of 
a  requirement 

Business 

Processes  and 
Operational 
Assessment: 
Analysis  of  time  to 
market  Req’ts 

SE  Architecture 
Req’ts:  Define  the 
types  of  Req’ts 

Business 

Processes  and 
Operational 
Assessment: 
Analysis  of 
product  platform 
alternatives 

SE  Architecture 
Req’ts:  Define  the 
types  and 
characteristics  of 
Req’ts  documents 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Business 

Processes  and 
Operational 
Assessment: 
Analysis  of  cost 
Req’ts_1 

SE  Architecture 
Req’ts:  Instruct 
the  integration  of 
SEA  Req’ts 
documents  with 
other  project 
documents 
(documentation 
tree)_1 

Business 

Processes  and 
Operational 
Assessment: 
Analysis  of 
behavioral  Req’ts 

SE  Architecture 
Req’ts:  Instruct 
the  Req’ts 
development 
method 

Business 

Processes  and 
Operational 
Assessment: 
Identify  what  is 
achievable  within 
the  cost,  schedule 
and  technical 
envelope 

SE  Architecture 
Req’ts:  Instruct 
the  Req’ts 
decomposition 
method 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

System,  Solution, 
Test  Architecture: 
Develop  Req’ts 

SE  Architecture 
Req’ts:  Student 
activity  to  exercise 
the  SEA  Req’ts 
definition 
methodology  to 
create  an 
unambiguous, 
testable,  and 
traceable  set  of 
Req’ts 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
determine  the 
customer’s 
operational  needs 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
gather  customer 
Req’ts  (6,  7) 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
allocate  Req’ts  to 
functional  and 
physical 
components  _1 

Req’ts  Mgmt 

Mgmt:  Risk, 
Configuration, 
Baseline:  Perform 
Req’ts  Mgmt 

System 

Engineering 

Architecture 

Method:  Instruct 
method  to 
establish  & 
manage  a  Req’ts 
baseline 

Business 

Processes  and 
Operational 
Assessment: 
Identify  and 
manage  solution 
functional  and 
operational 
baselines 

SE  Architecture 
Req’ts:  Instruct 
the  SEA  Req’ts 
Mgmt 

methodology/tools 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Functional 

Analysis  and 
Allocation 

Functional 

Analysis  and 
Allocation 

Functional 

Analysis  and 
Allocation 

System,  Solution, 
Test  Architecture: 
Develop 
functional 
architecture 

Architecture  and 
Design:  allocation 
of  functions  to 
meet  Req’ts 

System 

Engineering 

Architecture 

Method:  Instruct 
system  functional 
analysis; 
integration  of 
system 

operational  and 
support  functions; 
discussion  of 
related  methods 
(FFBDs,  IDEFs, 
N-Squared 

Charts) 

System,  Solution, 
Test  Architecture: 
Allocate  functions 
to  physical 
elements_1 

System 

Engineering 

Architecture 

Method:  Instruct 
the  concept  of 
system 
packaging; 
system 
architecture; 
allocation  of 
system-level 

Req’ts  to 
conceptual  sub¬ 
systems 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
allocate  Req’ts  to 
functional  and 
physical 
components  _2 

Physical  Solution/ 
Design  Synthesis 

Synthesis 

Synthesis 

System,  Solution, 
Test  Architecture: 
Develop  physical 
architecture 

Architecture  and 
Design:  systems 
architecture/synth 
esis 

System 

Engineering 

Architecture 

Method:  Instruct 
system 
architecture 
development 

System,  Solution, 
Test  Architecture: 
Allocate  functions 
to  physical 
elements_2 

Architecture  and 
Design: 

arrangement  of 
element, 
subsystems  and 
systems 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
allocate  Req’ts  to 
functional  and 
physical 
components  _3 

Architecture  and 
Design:  systems 
architecting 
process 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Systems  Analysis 

System  Analysis 
and  Control  Tools 

System 

Engineering 

Architecture 

Method:  Instruct 
the  concept  of 
system 
operational 
effectiveness 
(SOE),  and  the 
“cause-and-effect” 
analysis  between 
design,  and 
system  operation 
and  support 

System 

Engineering 

Architecture 

Method:  Instruct 
system 
architecture 
evaluation 

Tradeoff  Studies 

Trade  Studies 

Trade  Studies 

System,  Solution, 
Test  Architecture: 
Perform  trade 
studies 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
conduct  trade 
studies 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

M&S 

M&S 

Modeling, 
Simulation,  and 
Analysis:  Conduct 
system 
performance 
modeling  and 
forecasting 

Modeling, 
Simulation,  and 
Analysis:  Conduct 
system 
architecture 
modeling  and 
analysis 

Modeling, 
Simulation,  and 
Analysis:  Conduct 
operations 
analysis 

Modeling, 
Simulation,  and 
Analysis:  Conduct 
system  reliability, 
availability  and 
maintainability 
analysis_1 

Modeling, 
Simulation,  and 
Analysis:  Conduct 
system  safety 
analysis  _1 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Modeling, 
Simulation,  and 
Analysis:  Conduct 
system  user 
interface 
analysis_1 

Technical 

Performance 

Measures 

Technical 

Performance 

Measurement 

Technical 

Performance 

Measurement 

Mgmt:  Risk, 

Configuration, 

Baseline:  Manage 

technical 

performance 

measure 

variances 

SE  and 

Architecture 
Overview:  Instruct 
the  SEA  metrics 
and  technical 
performance 
measures 

Integration 

Mgmt:  Risk, 
Configuration, 
Baseline:  Conduct 
system  integration 

Integration: 
interfaces 
between  the 
elements_1 

System 

Engineering 

Architecture 

Method:  Instruct 
integration  with: 
Test  methods  and 
procedures; 
software 
development 
method;  service 
delivery  and  Mgmt 
method 

Integration:  test, 

verification, 

validation_1 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Verification 

Verification 

Mgmt:  Risk, 
Configuration, 
Baseline:  Conduct 
verification 

Validation 

Mgmt:  Risk, 
Configuration, 
Baseline:  Conduct 
validation 

Integration:  test, 

verification, 

validation_2 

SE  Mamt 

Mgmt:  Risk, 
Configuration, 
Baseline:  Manage 
schedule 
variances 

Introduction  to 

SE:  organization 
and  Mgmt  of 
complex 
systems_1 

SE  Process 
Environment 

Mgmt:  Risk, 
Configuration, 
Baseline:  Create 
the  SE 
organization 

Project  Mgmt  for 
Systems 

Engineers:  SE 
organization 

SE  and 

Architecture 
Overview:  Instruct 
the  typical  SEA 
project  staffing 
model 

Mgmt:  Risk, 
Configuration, 
Baseline:  Create 
and  manage  SE 
teams 

Introduction  to 

SE:  roles  and 
responsibilities 

Project  Mgmt  for 
Systems 

Engineers:  roles 
and 

responsibilities 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Introduction  to 

SE:  organization 
and  Mgmt  of 
complex 
systems_2 

SE  Planning  and 
Strategy 

SE  Planning 

Business 

Processes  and 

Operational 

Assessment: 

Design  the 
acquisition 
process 

Project  Mgmt  for 
Systems 

Engineers: 

planning 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
enable  technical 
planning  and 

Mgmt 

Mgmt:  Risk, 
Configuration, 
Baseline:  Develop 
and  manage 
processes 

Project  Mgmt  for 
Systems 

Engineers: 

scheduling 

SE  and 

Architecture 
Overview: 
Summarize  the 
plan  for  SEA 
training  and 
professional 
development_1 

Mgmt:  Risk, 
Configuration, 
Baseline:  Define 
the  SE  process 

Project  Mgmt  for 
Systems 

Engineers: 
resource  mgmt 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
enable  capacity 
planning_1 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Mgmt:  Risk, 
Configuration, 
Baseline:  Develop 
plans  and 
schedules 

Project  Mgmt  for 
Systems 

Engineers: 

process 

development 

Mgmt:  Risk, 
Configuration, 
Baseline:  Tailor 

SE  process 

Work  Breakdown 
Structure  (WBS) 

Work  Breakdown 
Structure  (WBS) 

Work  Breakdown 
Structure  (WBS) 

SEMP  (SE 
"Master"  Plan) 

SE  Architecture 
Req’ts:  Instruct 
the  integration  of 
SEA  Req’ts 
documents  with 
other  project 
documents 
(documentation 
tree)_2 

Risk  Mgmt 

Risk  Mgmt 

Risk  Mgmt 

Mgmt:  Risk, 
Configuration, 
Baseline:  Conduct 
risk  Mgmt 

Risk  Mgmt:  risk 
definition 

Mgmt:  Risk, 
Configuration, 
Baseline:  Manage 
key  drivers 

Risk  Mgmts: 
subsequent 
identification, 
assessment,  and 
Mgmt 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Configuration 

Mgmt 

Config.  Mgmt 

Config.  Mgmt 

Mgmt:  Risk, 
Configuration, 
Baseline:  Perform 
system 
configuration 

Mgmt 

Interface  Mgmt 

System,  Solution, 
Test  Architecture: 
Develop  interface 
architecture 

Integration: 
interfaces 
between  the 
elements_2 

System,  Solution, 
Test  Architecture: 
Develop  other 
system  interface 
architecture 

Modeling, 
Simulation,  and 
Analysis:  Conduct 
system  user 
interface 
analysis_2 

Manufacturing 
Readiness  Levels 

System,  Solution, 
Test  Architecture: 
Develop  the 
manufacturing 
system 
architecture 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Cost  Analysis 

Mgmt:  Risk, 
Configuration, 
Baseline:  Manage 
cost  variances 

Engineering 
Economics:  cost- 
benefit  analysis 

Business 

Processes  and 
Operational 
Assessment: 
Analysis  of  cost 
Req’ts_2 

Engineering 
Economics: 
introductory  level 
accounting 

Engineering 

Economics: 

budget 

fundamentals 

Lifecycle  Cost 
Analysis 

Engineering 
Economics:  life- 
cycle  cost 
estimation 

System 

Engineering 

Architecture 

Method:  Instruct 
system  life  cycle 
analysis  and  life 
cycle  costing 

Technical  Basis 
for  Cost 

Cost  Containment 

DoD  Acquisition 

Related  Topics 

Systems 

Acquisition 

Overview 

Acquisition  Policy 
and  the  Resource 
Allocation 

Process 

Source  Selection 
Planning 

Solicitations  and 
Source  Selection 

Solicitations  and 
Source  Selection 

F-47 


Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Architecture 

Frameworks 

Architecture  and 
Interoperability 
(from  TOC)/ 
Interoperability 
and  Architecture 
(from  Lesson 
Assignment 

Sheet) 

System,  Solution, 
Test  Architecture: 
Determine  and 
manage  impact  to 
current  fielded 
solutions 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
define  an 
architecture 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
enable  capacity 
planning_2 

Technology 
Insertion  Mgmt 

Technology 
Develop-ment  and 
Insertion 

Technology  Mgmt 

Mgmt:  Risk, 
Configuration, 
Baseline:  Analyze 
technology 
insertion  and 
refreshment 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Technical 

Reviews  amd 

Audits 

Technical 

Reviews 

Mgmt:  Risk, 
Configuration, 
Baseline:  Monitor 
and  control  of  a 
project 

SE  and 

Architecture 
Overview:  Define 
the  basic  content 
and  conduct  of  a 
System  Req’ts 
Review  (SRR), 
Preliminary 

Design  Review 
(PDR),  and 

Critical  Design 
Review  (CDR) 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
conduct  a  System 
Req’ts  Review 
(SRR) 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
conduct  a 
Preliminary 

Design  Review 
(PDR) 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

System 

Engineering 

Architecture 

Method:  Instruct 
the  method  to 
conduct  a  Critical 
Design  Review 
(CDR) 

Metrics 

System 

Engineering 

Architecture 

Method:  Instruct 
the  establishment 
and  Mgmt  of  SEA 
metrics  and 
Technical 
Performance 
Measures  (TPMs) 

SE  Enablers 

Process  Models 
and  Standards 

Introduction  to 

SE:  intro  to 
common  SE 
models 

SE  and 

Architecture 
Overview:  Instruct 
compliance  with 
SEI-CMM 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Introduction  to 

SE:  project 
lifecycles 

SE  and 

Architecture 
Overview:  Clarify 
and  explain 
relationship 
between  the  SEA 
methodology  and 
development 
models  such  as 
the  waterfall  and 
spiral  models_2 

Software  Design 
and  Development 

DoD  Software 
Acquisition 

Planning  and 
Strategy  (from 
TOC)/  The  DoD 
Software 

Acquisition  Mgmt 
Environment 
(from  Lesson 
Assignment 

Sheet) 

DoD  Software 
Acquisition 

Control  and 
Methodology 
(from  TOC)/  SW 
Engineering 
Process  (from 
Lesson 

Assignment 

Sheet) 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

DoD  Software 
Acquisition 

Planning  and 
Strategy  (from 
TOC)/  SW  Best 
Practices  (from 
Lesson 

Assignment 

Sheet) 

Software  Mgmt 
Exercise  (Case 
Scenario  and 
Workshop) 

IPPD 

Integrated 

Product  and 

Process 

Development 

Integrated 

Product  and 

Process 

Development 

Integrated 

Product  and 

Process 

Development 

Product 

Improvement 

Product 

Improvement 

Production:  IPPD 
and  the  Paladin 
Enterprise_1 

SE  Desiqn 
Considerations 

Training 

System,  Solution, 
Test  Architecture: 
Develop  the 
training  system 
architecture 

SE  and 

Architecture 
Overview: 
Summarize  the 
plan  for  SEA 
training  and 
professional 
development_2 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Serviceability, 
Logistics:  Develop 
system  training  l 

Reliability, 

Availability, 

Maintain-ability 

Modeling, 
Simulation,  and 
Analysis:  Conduct 
system  reliability, 
availability  and 
maintainability 
analysis_2 

Reliability  and 
Maintainability: 
how  to  develop  a 
systems 

architecture,  from 
conceptual  design 
onward,  that 
meets  the  user's 
reliability  and 
maintainability 
concerns 

Serviceability, 
Logistics:  Develop 
system 
maintenance 
concepM 

Test  and 

Evaluation 

System,  Solution, 
Test  Architecture: 
Develop  test 
system 
architecture 
(functional, 
physical  and 
interface) 

Integration:  test, 

verification, 

validation_3 

Producability  & 
Manufactur-ability 

Production:  IPPD 
and  the  Paladin 
Enterprise_2 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Sustainability 

Deployment, 
Operation  and 
Support_1 

Supportability/ 

Acquisition 

Logistics 

Deployment, 
Operation  and 
Support_2 

System,  Solution, 
Test  Architecture: 
Develop  the 
deployment 
system 
architecture 

Logistics  and 
Supportability 

Serviceability, 
Logistics:  Develop 
system 
maintenance 
concept_2 

Serviceability, 
Logistics:  Develop 
system  supply 
support  and 
spares  Mgmt 

Serviceability, 
Logistics:  Analyze 
system 

operational  and 
servicing  skill 

Req’ts 

Serviceability, 
Logistics:  Develop 
system  and 
platform 
documentation 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Serviceability, 
Logistics:  Develop 
system  train ing_2 

ESOH 

Environ-mental, 
Safety  and  Health 

Environmental, 
Safety,  and 
Occupational 

Health 

Human  Systems 
Integration 

System,  Solution, 
Test  Architecture: 
Develop  user  / 
operator  / 
maintainer 
interface 
architectures 

Safety  and 

Security 

Modeling, 
Simulation,  and 
Analysis:  Conduct 
system  safety 
analysis  _2 

Other 

Admin 

Concept  and 

Technology 

Development 

Mgmt:  Risk, 
Configuration, 
Baseline:  Select 
the  SE  tools 

System 

Engineering 

Architecture 

Method:  Provide  a 
complex  system 
development  case 
study  and 
“lessons  learned” 
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Source  Title 

DAU  SPRDE  SYS 
201A 

DAU  SPRDE  SYS 
201 B 

DAU  ASPRDEC 
301 

Boeing 

National 

Cryptologic 

School 

IBM  Req’ts 

Benefits  and 
Challenges  of 
International 
Cooperation 

System  Definition 

Design, 

Fabrication  and 
Test 

Current  Events 
and  Issues 

Improvements  to 
Existing  Weapon 
Systems 

Professional 

Ethics  Case 
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SE  TOPICS:  CERTIFICATION  PROGRAMS  (1) 


Sources  for  this  table: 

•  Aerospace  Corporation/  Aerospace  Initiative 

•  California  Institute  of  Technology  (Cal  Tech) 

•  INCOSE 

•  Massachusetts  Institute  of  Technology  (MIT) 

•  Naval  Post  Graduate  School 

•  Portland  State  University 


Source 

Information 

Aerospace 

Corporation 

Cal  Tech 

INCOSE 

MIT 

Naval  Post 
Graduate  School 

Portland  State  U. 

SE  Process 

SE 

Understanding 
Systems  and  SE 

General  SE 
Knowledge 

SE  Methods 

SE  and 
Architecture^ 

SE  Approach 

Req’ts 

Development 

Constructing  SE 
Req’ts 

Req’ts  and 

Architecture 

Definitionl 

Req’ts 

Engineering 

Functional  Analysis 
and  Allocation 

Performing  a 
Functional 

Analysis 

Physical  Solution/ 
Design  Synthesis 

Developing  a 

systems 

architecture 

System 

Architecture 

Performing 

System  Design 
and  Development 

System  Behaviors 
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Source 

Information 

Aerospace 

Corporation 

Cal  Tech 

INCOSE 

MIT 

Naval  Post 
Graduate  School 

Portland  State  U. 

Systems  Analysis 

Systems  Analysis 

Systems 

Dynamics  and 
Systems  Thinking 

M&S 

Modeling  and 
Simulation 

Business 

Modeling  and 
Simulation 

Discrete 

Multivariate 

Modeling 

Integration 

Systems 

Integration, 
Verification,  and 
Validation_1 

Hardware  and 

Software 

Integration^ 

Verification 

Verification  and 
Validation 

Testing_1 

Systems 

Integration, 
Verification,  and 
Validation_2 

Validation 

Verification  and 
Validation 

Testing_2 

Systems 

Integration, 
Verification,  and 
Validation_3 

SE  Mqmt 

SE  Mgmt 

Program  Mgmt 

Operations 
Research  in 
Engineering  Mgmt 

SE  Process 
Environment 

Aerospace  Roles 
in  Space 

Systems 
Architecting, 
Acquisition,  and 
Engineering 

Engineering 
Process  Mgmt 

Organizational 

Processes 
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Source 

Information 

Aerospace 

Corporation 

Cal  Tech 

INCOSE 

MIT 

Naval  Post 
Graduate  School 

Portland  State  U. 

Managing 

Changes  in  Work 
and  Organizations 

SE  Planning  and 
Strategy 

SE  Mgmt  and 
Planning 

Managing  System 
Cost  and 

Schedule 

Estimation_2 

Risk  Mgmt 

Managing  Risk 

Configuration 

Mgmt 

Using 

Configuration 

Mgmt 

Cost  Analysis 

Cost 

Managing  System 
Cost  and 

Schedule 

Estimation_2 

Cost  Analysis 

DoD  Acquisition 

Related  Topics 

Source  Selection 

SE  Indicators 

Architecture 

Frameworks 

Architecting 

Req’ts  and 

Architecture 

Definition_2 

SE  and 

Architecture_2 

Technology 

Insertion  Mgmt 

Managing 
Technology  and 
Innovation 
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Source 

Information 

Aerospace 

Corporation 

Cal  Tech 

INCOSE 

MIT 

Naval  Post 
Graduate  School 

Portland  State  U. 

Technical  Reviews 
and  Audits 

Conducting 

Technical 

Reviews  and 

Audits 

SE  Enablers 

Software  Design 
and  Development 

Hardware  and 

Software 

lntegration_2 

IPPD 

Creating  a  High 
Performing  Team 
for  SE_2 

Communicating  in 
Groups  and 

Teams 

SE  Desiqn 
Considerations 

Reliability, 

Availability, 

Maintainability 

Reliability 

Engineering 

Test  and 

Evaluation 

Verification  and 
Validation 

Testing_3 

Producability  & 
Manufacturability 

Producing  the 
system 

Manufacturing 

Systems 

Simulation 

Supportability/ 

Acquisition 

Logistics 

Logistics  SE 

Safety  and  Security 

Security  systems 
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Source 

Information 

Aerospace 

Corporation 

Cal  Tech 

INCOSE 

MIT 

Naval  Post 
Graduate  School 

Portland  State  U. 

Other 

Computer 

Systems 

Communications 

Systems 

Satellite  Systems 
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SE  TOPICS:  CERTIFICATION  PROGRAMS  (2) 


Sources  for  this  table: 


•  Reliability  Analysis  Center  (RAC)/ American  Society  of  Naval  Engineers  (ASNE)/Advanced  Automating 
Corporation  (AAC) 

•  Rensselaer  Polytechnic  Institute  (RPI) 

•  Southern  Methodist  University  (SMU) 

•  Southern  Polytechnic  State  University 

•  Stevens  Institute  of  Technology 

•  University  of  Alabama,  Huntsville  (UAH)  Professional  Development  SE  Program 


Source 

Information 

RAC  /  ASNE  /  AAC 

RPI 

SMU 

Southern 
Polytechnic 
State  U. 

Stevens  Institute 
of  Technology 

U  of  Huntsville, 
Alabama 

SE  Process 

SE  Process 

Introduction  to 

SE 

Systems 

Engineering 

Req’ts 

Development 

Req’ts 

Development 

Physical  Solution/ 
Design  Synthesis 

SE:  Principles  and 
Implementation: 

Product  Realization 

Systems  Design 

System  Analysis 
and  Systems 
Design_1 

Systems 
Architecture  and 
Design 

System 

Architecture 

Systems  Analysis 

Maintenance 

Engineering:  Principles 
and  Applications: 
Analysis_1 

Systems 

Analysis 

Methods 

System  Analysis 
and  Systems 
Design_2 

Operational 
Effectiveness  and 
Life-Cycle 
Analysis_1 
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Source 

Information 

RAC /  ASNE /  AAC 

RPI 

SMU 

Southern 
Polytechnic 
State  U. 

Stevens  Institute 
of  Technology 

U  of  Huntsville, 
Alabama 

SE:  Principles  and 
Implementation: 

Analysis  and  Evaluation 

Systems 

Analysis  and 
Optimization 

Systems 

Analysis, 
Modification  and 
Simulation 

M&S 

Modeling  and 
simulation 

Simulation  and 
Modeling 

Integration 

Systems 
Integration  and 
Testing_1 

Verification 

Verification 
Program 
Development 
and  Mgmt_1 

Systems 

Validation  and 
Verification^ 

Validation 

Systems 

Validation  and 
Verification_2 

SE  Mqmt 

SE:  Principles  and 
Implementation: 

Technical  Mgmt 

Managing  the 
Technical  Effort 
Associated  with 
Systems 

Creation 

Product  Mgmt  of 
Complex  Systems 

Program  and 
System  Mgmt 

Verification 
Program 
Development 
and  Mgmt_2 
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Source 

Information 

RAC /  ASNE /  AAC 

RPI 

SMU 

Southern 
Polytechnic 
State  U. 

Stevens  Institute 
of  Technology 

U  of  Huntsville, 
Alabama 

SE  Planning  and 
Strategy 

Competitive 
Advantage  and 
Operations 
Strategy 

Decision  and  Risk 
Analysis  for 
Complete 

Systems 

Decision  Making 

Risk  Mgmt 

Integrated  Risk 
Mgmt 

Risk  Mgmt 

Configuration 

Mgmt 

SE:  Principles  and 
Implementation: 

Product  Control 

Advanced 

Configuration 

Mgmt 

SE:  Principles  and 
Implementation: 
Configuration  and  Data 
Mgmt_1 

Data  Mgmt 

SE:  Principles  and 
Implementation: 
Configuration  and  Data 
Mgmt_2 

Engineering 

Economic 

Analysis 

Lifecycle  Cost 
Analysis 

Operational 
Effectiveness  and 
Life-Cycle 
Analysis_2 

SE  Enablers 
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Source 

Information 

RAC /  ASNE /  AAC 

RPI 

SMU 

Southern 
Polytechnic 
State  U. 

Stevens  Institute 
of  Technology 

U  of  Huntsville, 
Alabama 

Process  Models 
and  Standards 

SE:  Principles  and 
Implementation:  SE 
Standards 

SE:  Principles  and 
Implementation:  Models 

Software  Design 
and  Development 

Specialty  Engineering 
for  Product  Support: 
Systems,  Applications, 
and  Integration: 

Software  Engineering 

Software  SE 

Software 

Product  Mgmt 

IPPD 

Process 

Assessment  and 
Improvement 

SE  Desiqn 
Considerations 

Training 

Specialty  Engineering 
for  Product  Support: 
Systems,  Applications, 
and  Integration: 

Training  Systems 
Development 

Reliability, 

Availability, 

Maintainability 

Specialty  Engineering 
for  Product  Support: 
Systems,  Applications, 
and  Integration: 

Reliability  Engineering 

Reliability 

Engineering 

F-65 


Source 

Information 

RAC /  ASNE /  AAC 

RPI 

SMU 

Southern 
Polytechnic 
State  U. 

Stevens  Institute 
of  Technology 

U  of  Huntsville, 
Alabama 

Specialty  Engineering 
for  Product  Support: 
Systems,  Applications, 
and  Integration: 
Maintainability 
Engineering 

Maintenance 

Engineering:  Principles 
and  Applications: 
Maintenance 

Predictions 

Maintenance 

Engineering:  Principles 
and  Applications: 
Maintenance  Design 
Methods 

Maintenance 

Engineering:  Principles 
and  Applications: 
Operational  SE 

Principles  and 
Applications 

Maintenance 

Engineering:  Principles 
and  Applications: 
Analysis_2 
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Source 

Information 

RAC /  ASNE /  AAC 

RPI 

SMU 

Southern 
Polytechnic 
State  U. 

Stevens  Institute 
of  Technology 

U  of  Huntsville, 
Alabama 

Test  and 

Evaluation 

Specialty  Engineering 
for  Product  Support: 
Systems,  Applications, 
and  Integration: 
Testability  Engineering 

Systems 
Integration  and 
Testing_2 

Maintenance 

Engineering:  Principles 
and  Applications: 

Testing  and 
demonstration 

Producability  & 
Manufacturability 

Specialty  Engineering 
for  Product  Support: 
Systems,  Applications, 
and  Integration: 
Manufacturing 
Engineering 

Analysis  of 

Manufacturing 

Processes 

Design  for 

Systems 

Reliability, 
Maintainability, 
and  Supportability 

Manufacturing 

Systems 

Integration 

Manufacturing 
Systems  Mgmt 

Supportability/ 

Acquisition 

Logistics 

Specialty  Engineering 
for  Product  Support: 
Systems,  Applications, 
and  Integration: 
Supportability  Analysis 

Logistics  SE 

System 

Supportability  and 
Logistics 
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Source 

Information 

RAC /  ASNE /  AAC 

RPI 

SMU 

Southern 
Polytechnic 
State  U. 

Stevens  Institute 
of  Technology 

U  of  Huntsville, 
Alabama 

SE:  Principles  and 
Implementation: 

Product  Support 

Human  Systems 
Integration 

Specialty  Engineering 
for  Product  Support: 
Systems,  Applications, 
and  Integration:  Human 
Factors  Engineering 

Other 

Specialty  Engineering 
for  Product  Support: 
Systems,  Applications, 
and  Integration: 
Information  Technology 
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SE  TOPICS:  CERTIFICATION  PROGRAMS  (3) 


Sources  for  this  table: 

•  University  of  Arizona 

•  University  of  California  (UC),  Riverside 

•  University  of  California  (UC),  San  Diego 

•  University  of  Minnesota 

•  University  of  Mississippi— Rolla/  University  of  Southern  California  (USC) 

•  University  of  Southern  California  (USC) 

•  Wichita  State  University 


Source 

Information 

U.  of  Arizona 

UC,  Riverside 

UC,  San  Diego 

U.  of 

Minnesota 

U.  of 

Mississippi-- 
Rolla /  USC 

USC 

Wichita  State 

U. 

SE  Process 

SE  Process 

Principles  of 
SE 

SE  Theory  and 
Practice 

SE  Practices 

1  and  II 

Advanced 

Topics  in  SE 

Req’ts 

Development 

System  Req’ts 
Development 
and  Analysis 

Systems  Req’ts 
Analysis 

Systems 

Software  Req’ts 
Engineering^ 
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Source 

Information 

U.  of  Arizona 

UC,  Riverside 

UC,  San  Diego 

U.  of 

Minnesota 

U.  of 

Mississippi-- 
Rolla /  USC 

USC 

Wichita  State 

U. 

Systems  Req’ts 
Analytical 
Techniques  and 
Tools 

Functional 
Analysis  and 
Allocation 

System 

Concept 

Development 

and 

Selection_1 

Concept 

Development_1 

Physical 

Solution/ 

Design 

Synthesis 

Model-Based 

Systems 

Design 

System 

Concept 

Development 

and 

Selection_2 

Concept 

Development_2 

Systems 

Architecting 

Systems 

Architecting 

System  Design 
and 

Integration^ 

Systems 

Analysis 

Simulation, 
Modeling  and 
Analysis_1 

Engineering 

Analysis 

Statistical 
Methods  for 
Engineers 

Engineering 

statistics 

SE  and 

Analysis 

Optimization 

Methods 

Linear  Systems 
Theory 
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Source 

Information 

U.  of  Arizona 

UC,  Riverside 

UC,  San  Diego 

U.  of 

Minnesota 

U.  of 

Mississippi-- 
Rolla /  USC 

USC 

Wichita  State 

U. 

M&S 

Simulation, 
Modeling  and 
Analysis_2 

SE  Solutions 

Using  Adaptive 

Rule-Based 

Simulation 

Modeling  and 
Simulation 

Integration 

System  Design 
and 

lntegration_2 

Systems 

Hardware  and 

Software 

Integration^ 

Verification 

Systems 

Verification 

Systems 

Validation  and 
Verification^ 

Validation 

Systems 

Validation  and 
Verification_2 

SE  Mqmt 

SE  Mgmt 

SE  Mgmt 

SE  Mgmt 

Mgmt 

Engineering 
Project  Mgmt 

Engineering 

Mgmt 

Program  Mgmt 
Essentials 

Analysis  of 

Decision 

Processes 

SE  Process 
Environment 

Program 

Manager  Boot 
Camp 

Mgmt  of 

Engineering 

Teams 

Cost  Analysis 

Engineering 

Economy 
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Source 

Information 

U.  of  Arizona 

UC,  Riverside 
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IDENTIFIED  SYSTEMS  ENGINEERING  BEST  PRACTICES  OR  DEFICIENCIES 


Source 

Best  Practice  or  Deficiency 

Recommendations 

GAO  Reports 

Strong  Management,  Processes, 
and  Metrics  Needed  to  Improve 
Software  Acquisition,  Pending 
Release 

Working  in  a  manageable  evolutionary 
environment 

To  assure  DoD  appropriately  sets  and  manages 
requirements,  acquirers  should  obtain  signed 
agreements  with  the  contractor  to  confirm  software 
requirements  based  on  knowledge  obtained  through 
the  completion  of  systems  engineering  requirements 
analysis  and  require  cost/benefit  analysis  for  proposed 
major  requirements  changes 

Following  a  disciplined  development  process 

To  ensure  DoD  acquisitions  are  managed  to  a 
disciplined  process,  acquirers  should  develop  a  list  of 
software  knowledge  deliverable  based  on  the 
completion  of  systems  engineering  that  contractors  are 
required  to  provide  at  the  end  of  software  phases: 
requirements,  design,  coding,  and  testing. 

Collecting  and  analyzing  meaningful  metrics 

To  ensure  DoD  has  the  knowledge  it  needs  to  oversee 
intensive  acquisitions,  acquirers  should  require 
software  contractors  to  collect  and  report  metrics 
related  to  cost,  schedule,  size,  requirements,  tests, 
defects,  and  quality  to  program  offices  on  a  monthly 
basis  and  before  program  milestones 

Acquirers  should  ensure  that  contractors  have  an 
earned  value  management  system  that  reports  cost 
and  schedule  information  at  a  level  of  work  that 
provides  information  specific  to  software  development. 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Defense  Acquisition:  Employing 

Best  Practices  Can  Shape  Better 
Weapons  Decisions,  April  2000 

A  knowledge-based  approach,  based  on 
commercial  best  practices,  should  shape  DoD 
acquisition,  at  three  knowledge  points: 

•  When  technology  meets  requirements 

•  When  the  design  meets  performance 
standards 

•  When  the  product  can  be  produced  within 
cost,  schedule,  and  quality  targets 

In  initial  stages  of  product  development: 

•  Instill  flexibility  in  both  resources  provided  and 
the  product’s  performance  requirements  to  allow  for 
the  uncertainties  of  technical  progress 

•  Instill  high  standards  for  technical  maturity 

Defense  Acquisition:  Employing 

Best  Practices  Can  Shape  Better 
Weapons  Decisions,  April  2000, 
cont. 

Undertake  product  launch  when  technology  matures 
enough  to  meet  requirements,  as  measured  by 
technology  readiness  levels  (TRLs) 

Develop  estimates  of  cost  and  schedule  and  finalize 
requirements  only  when  90%  of  the  engineering 
drawings  have  been  finalized 

Initiate  full  rate  production  only  when  it  is  proved  that 
the  product  can  meet  cost,  schedule,  and  quality 
targets 

Better  Matching  of  Needs  and 
Resources  Will  Lead  to  Better 
Weapons  Systems  Outcomes, 

March  2001 

Customer’s  needs  and  developer’s  resources 
need  to  be  matched  early  in  the  process  by: 

•  Using  systems  engineering  at  the 
beginning  of  the  process,  preferable  before  it 

Do  systems  engineering  early 

Systems  engineering  contracts  should  be  made  with 
developers  before  the  systems  development  and 
demonstration  phase 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

begins 

•  Customer  and  developer  being  flexible 
about  making  trade-offs  and  lowering 
expectations  (e.g.,  deferring  additional 
capabilities  to  the  next  generation  of  a  product 
if  including  them  in  the  present  generation 
would  substantially  increase  costs  or  would 
involve  using  unproven,  immature  technology 

•  Customer  and  developer  having  equal 
roles  in  the  process,  sharing  responsibility  for 
setting  requirements 

Pressure  to  set  unrealistic  requirements  should  be 
reduced  via  top-level  involvement 

A  Constructive  Test  Approach  is 

Key  to  Better  Weapon  Systems 
Outcomes,  July  2000 

Best  practice  in  testing  and  evaluation  are  to  test 
early  and  use  those  tests  to  validate  knowledge- 
use  tests  to  learn  from  rather  than  as  a  scorecard 
for  funding 

Program  mangers  and  testers  should  cooperate  in 
defining  and  scheduling  tests  around  product  maturity 
levels 

Specific  maturity  levels  should  be  attained  before  major 
program  approval 

Software  and  Systems  Process 
Improvement  Programs  Vary  in 

Use  of  Best  Practices,  March  2001 

Lack  of  SPI  implementation  in  DoD  components 

Implement  software  process  improvement  (SPI) 
programs  where  none  exist 

No  DoD  knowledge-sharing  activities  of  best- 
practice  examples  of  successful  SPI  programs 
despite  the  fact  that  components  with  SPI 
programs  reported  higher  productivity,  higher 
product  quality,  and  higher  ROI 

Issue  IDEAL-based  guidance  on  SPI  (IDEAL  is  SEI’s 
model  of  Initiating,  Diagnosing,  Establishing,  Acting, 
and  Leveraging) 

DoD  has  not  actively  promoted  and  leveraged  SPI 
programs  due  to  resource  constraints,  doubts 
about  SPI  costs  and  benefits,  and  other  priorities 

Perform  an  annual  SPI-compliance  assessment  and 
develop  DoD-wide  activities  to  disseminate  knowledge 
and  best  practices  on  SPI 

DoD  Teaming  Practices  Not 
Achieving  Potential  Results,  April 
2001 

IPTs  are  more  efficient  way  in  which  to  develop  a 
product  or  system 

Give  PMs  more  responsibility  for  the  product  (weapons 
system  program)  and  more  decision-making  authority 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Most  effective  IPTs  have  members: 

•  Who  represent  a  cross-section  of  the 
expertise  needed  to  develop  a  product 

•  Are  collocated 

•  Are  selected  by  the  team  leader 

Designate  as  IPTs  only  those  teams  that  will  have  the 
day-to-day  responsibility  for  developing  and  delivering 
a  product  and  the  expertise  to  do  so 

Team  members,  along  with  the  leader,  have 

•  Responsibility  for  delivering  the  product 

•  Authority  to  make  decisions 

•  The  required  expertise 

Establish  a  culture  supportive  of  IPTs  through  a  system 
of  professional  education  for  program  managers  and 

IPT  leaders 

Capturing  Design  and 

Manufacturing  Knowledge  Early 
Improves  Acquisition  Outcomes, 

July  2002 

Capture  design  and  manufacturing  knowledge 
prior  to  two  key  decision  points: 

•  When  the  product  design  is  determined  to 
be  capable  of  meeting  product  requirements 
and  is  stable 

•  When  a  reliable  product  can  be  produced 
repeatedly  and  at  an  affordable  cost 

Design  is  stable: 

•  Limit  the  design  challenge 

•  Demonstrate,  through  prototyping  or  other 
means,  that  product  works 

•  Complete  design  review  of  system  and 
subsystems 

•  Obtain  stakeholder  concurrence  that  the  design 
is  complete  and  producible 

Keep  the  design  challenge  manageable  by 
adopting  an  evolutionary  approach  that  minimizes 
technology  changes  being  made  at  any  one  time 

Product  can  be  produced: 

•  Identify  key  system  characteristics  and  critical 
manufacturing  processes 

•  Determine  that  processed  are  in  control  and 
are  stable 

•  Analyze  potential  failure  modes  and  their 
effects  on  performance 

Separate  technology  from  product  development 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Better  Acquisition  Outcomes  are 
Possible  if  DoD  Can  Apply 

Lessons  from  F/A-22  Program, 

April  11, 2003 

Evolutionary  product  development 

Knowledge  point  1  (should  occur  before  product 
launch): 

•  Separate  technology  from  product 
development 

•  Have  clear  measures  and  high  standards  for 
assessing  technology  maturity —  technology 
readiness  levels 

•  Use  a  disciplined  systems  engineering  process 
for  translating  and  balancing  customer’s  desires  with 
product  developer’s  technology,  design,  and 
production  limitations;  in  other  words,  bring  the  right 
knowledge  to  the  table  when  laying  down  a 
program’s  foundation 

•  Identify  the  mismatches  between  desired 
product  features  and  the  product  developer’s 
knowledge  and  either  (1 )  delay  the  start  of  the  new 
product  development  until  knowledge  deficit  can  be 
made  up  or  (2)  reduce  product  features  to  lessen 
their  dependency  on  areas  where  knowledge  is 
insufficient  (evolutionary  acquisition).  The  main 
opportunities  for  trading  of  design  features  to  save 
time  and  money  occur  here,  before  a  program  is 
started 

•  When  do  you  know  you  have  achieved  this 
knowledge  points?  When  technologies  needed  to 
meet  essential  product  requirements  have  been 
demonstrated  to  work  in  their  intended  environment 
and  the  producer  has  completed  a  preliminary  design 
of  the  product 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Better  Acquisition  Outcomes  are 
Possible  if  DoD  Can  Apply 

Lessons  from  F/A-22  Program, 

April  11, 2003,  cont. 

Knowledge-based  product  development  process: 

•  Match  between  customer  needs  and 
available  resources 

•  Demonstrate  ability  of  product  design  to 
meet  requirements  and  is  stable 

•  Prove  that  product  can  be  manufactured 
given  cost,  time  and  quality  constraints 

Knowledge  point  2  (should  occur  midway  between 
system  integration  and  demonstration: 

•  Hold  a  major  decision  review  between  system 
integration  and  system  demonstration  that 
determines  that  the  product  design  is  stable  and 
includes  specific  criteria  to  move  into  the  system 
demonstration  phase 

•  Use  integrated  engineering  prototypes  to 
demonstrate  design  stability  and  prove  with  testing 
that  the  design  meets  the  customer’s  requirements.  It 
is  important  that  this  happen  before  initial 
manufacturing  begins —  a  point  when  investments 
are  increased  to  produce  an  item 

•  Identify  critical  manufacturing  processes  and 
establish  a  plan  to  bring  these  under  statistical 
control  by  the  start  of  production;  also  establish 
reliability  goals  and  growth  plan  to  achieve  these  by 
production.  This  facilitates  the  achievement  of 
process  control  and  reliability  goals  at  the  completion 
of  knowledge  point  3 

•  When  do  you  know  you  have  achieved  this 
knowledge  point?  When  90  percent  of  engineering 
drawings  are  releasable  to  manufacturing 
organizations.  Drawings  are  the  language  used  by 
engineers  to  communicate  to  the  manufacturers  the 
details  of  the  new  product —  what  it  looks  like,  how  its 
components  interface,  how  to  build  it  and  the  critical 
materials  and  processes  needed  to  fabricate  it.  This 
makes  drawings  a  key  measure  of  whether  the 
design  is  stable  or  not 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Better  Acquisition  Outcomes  are 
Possible  if  DoD  Can  Apply 

Lessons  from  F/A-22  Program, 

April  11, 2003,  cont. 

Knowledge  point  3  (should  occur  before  production): 

•  Demonstrate  that  all  critical  manufacturing 
processes  are  under  statistical  control  and 
consistently  producing  items  within  the  quality 
standards  and  tolerances  for  the  overall  product 
before  production  begins.  This  is  important,  since 
variation  in  one  process  can  reverberate  to  others 
and  result  in  defective  parts  that  need  to  be  repaired 
or  reworked 

•  Demonstrate  product  reliability  before  the  start 
of  production.  This  requires  testing  to  identify  the 
problems,  design  corrections,  and  retest  the  new 
design.  Commercial  firms  consider  reliability 
important  and  its  achievement  a  measure  of  design 
maturity 

•  When  do  you  know  you  have  achieved  this 
knowledge  point?  When  all  key  manufacturing 
processes  have  come  under  statistical  control  and 
product  reliability  has  been  demonstrated 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Better  Acquisition  Outcomes  are 
Possible  if  DoD  Can  Apply 

Lessons  from  F/A-22  Program, 

April  11, 2003,  cont. 

Cultural  changes: 

•  Top  level  needs  to  take  control  of  investment 
dollars  and  say  “No”  to  programs  as  necessary 

•  Keep  key  personnel  in  place  long  enough  to 
affect  decisions  and  accountability 

•  Provide  programs  with  the  necessary  skilled 
people  to  craft  acquisition  approaches  and  over  see 
a  contractor’s  execution  of  the  program 

•  Realign  funding  between  S&T  and  acquisition 
organizations  to  separate  technology  from  product 
development 

•  Bring  discipline  to  the  requirements  process  by 
insisting  on  a  match  between  requirements  and 
resources 

•  Require  readiness  and  operating  cost  as  KPPs 
before  beginning  acquisition 

•  Design  and  implement  test  programs  to  provide 
the  knowledge  necessary  when  needed 

DSB  Reports 

Report  of  the  Defense  Science 
Board  Task  Force  on  Defense 
Software,  November  2000 

Stress  Past  Performance  and  Process  Maturity 

Initiate  Independent  Expert  Reviews  (lERs) 

Improve  Software  Skills  of  Acquisition  and  Program 
Management 

Collect,  Disseminate,  and  Employ  Best  Practices 

Restructure  Contract  Incentives 

Strengthen  and  Stabilize  the  Technology  Base 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study  Report  on  the  Health  of  SE  Process  Execution 

Requirements  Management 

Incomplete  definition  of  requirements 

Enforce  review  of  the  operational  requirements 
document  at  Milestone  B  and  do  not  proceed  past  the 
MB  if  the  operational  requirements  definitions  are 
incomplete,  not  approved,  or  not  signed  by  the 
appropriate  approving  authorities.  Require  a  complete 
and  signed  ConOps  document  at  MB.  Require  the 
program  to  address  how  the  documented  operational 
requirements  meet  the  Operational  Views  and  System 
Views  that  include  the  system. 

Establish  a  process  to  assure  the  proposal  solicitation 
System  Requirements  Document  (RD)  is  complete  and 
accurate.  For  major  programs,  establish  an 
independent  review  process  to  assure  the 
completeness  and  accuracy  of  the  SRD.  Require  the 
program  to  address  how  the  SRD  requirements  will 
meet  the  Operation  Views  and  System  Views 
requirements. 

Reinforce  the  requirement  that  the  IMP  must  have  well- 
defined  accomplishments  with  entrance  and  exit  criteria 
for  program  milestones  and  events.  The  Systems 
Requirement  Review  (SRR)  must  be  a  key  event  in  the 
IMP  with  exit  criteria  that  assure  the  system 
specification  is  complete,  including  a  complete  set  of 
verification  requirements.  The  SRR  must  be  the  gate 
which  is  opened  to  proceed  into  design  with  a  complete 
set  of  requirements  defined  or  held  closed  until  the 
system  level  requirements  are  defined. 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 

Requirements  creep 

The  system  program  director/manager  writes  a 
program  directive  instructing  all  program  stakeholders 
to  refrain  from  verbally  directing  changes  to  the 
contractor.  All  program  stakeholders  should  be 
instructed  not  to  verbally  direct  the  contractor  to 
implement  changes.  The  contractor  should  be  directed 
to  not  accept  and  act  on  verbally  directed/requested 
changes. 

Establish  written  guidance  on  the  role  of  government 
personnel  when  participating  in  Contractor’s  IPTs. 
Include  a  list  of  “Do’s  and  Don’ts.” 


Require  agreement  with  stakeholders  that  requirements 
growth  will  be  properly  managed  buy  changing  program 
baselines.  Educate  stakeholders  on  impact  of  creeping 
requirements;  how  it  is  counter  to  rapid  fielding  of 
operational  capability. 

Establish  program  plans  to  accommodate  requirements 
changes  that  exceed  the  program  baseline  by  deferring 
them  to  future  upgrades,  modifications,  or  increments. 
Alternatively,  establish  formal  change  to  the  baseline 
program  with  adjustment  of  the  program  schedule  and 
costs. 

Establish  a  process  to  formally  solicit  and  process  all 
suggested  requirements  changes.  This  process  should 
include  the  leadership  of  the  acquisition  organization  to 
assure  the  requirements  changes  are  reviewed  and 
managed  in  accordance  with  the  acquisition/contracting 
rules  of  engagement. 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 

Requirements  creep,  cont. 

Contractually  rebaseline  the  program  in  terms  of 
performance,  cost,  and  schedule  when  requirements 
changes  place  the  newly  defined  program  at  risk  1  the 
executing  within  the  original  performance,  cost,  and 
schedule  baselines. 

Address  requirements  identified  subsequent  to  the 
SRR  in  future  upgrades.  Place  strict  limits  on  minor 
requirements  changes  that  are  incorporated  into  the 
baseline  without  adjusting  the  program  baseline. 

Requirements  volatility  Implement  program  metrics  to  measure  and  track  the 

extent  of  requirements  changes  in  terms  of  number  of 
requirements  “shall  statements”  changed  from  contract 
award  forward  through  the  development  cycle.  This 
should  be  tracked  for  each  baseline  specification.  This 
would  constitute  a  program  metric  that  is  reported  and 
reviewed  periodically.  If  requirements  changes  continue 
to  grow  significantly  as  the  system  design  development 
progresses,  program  officials  should  expect  the  scope 
of  work  required  to  complete  the  program  to  grow 
beyond  the  program  cost  and  schedule  baseline. 

Readjust  the  program  schedule  and  cost  baselines  if 
the  level  of  requirements  changes  becomes  excessive 
to  accommodate  the  increased  work  generated  by  high 
level  of  change  activity. 

It  may  not  be  prudent  to  rebaseline  the  program  to 
accommodate  the  rate  of  requirements  changes, 
especially  if  they  are  externally  driven.  In  this  case, 
consideration  should  be  given  to  accommodating  the 
changes  in  follow  on  efforts  (contract  changes  to 
handle  new  requirements  in  later  increments,  upgrades 
and  modifications). 
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Source 


Best  Practice  or  Deficiency 

Dayton  Aerospace  Quick  Study  Lack  of  a  specification  tree 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 


Incomplete/weak  verification  requirements 


Recommendations 

Establish  a  requirement  in  proposal  solicitations  for 
delivery  of  a  preliminary  specification  tree  and 
description  of  the  systems  engineering  processes  that 
will  be  used  to  maintain  the  specifications  and 
specification  tree  in  the  proposal,  e.g.,  in  the  section  L 
instructions  to  the  offeror.  Proposers  must  include  a 
preliminary  specification  tree  with  the  proposal  and 
their  proposed  systems  engineering  processes  must 
include  requirements  to  maintain  and  update  the 
specification  tree  over  the  life  of  the  development 
program.  This  includes  maintenance  of  specifications 
that  are  not  under  government  control  but  are 
necessary  to  maintain  the  integrity  of  the  development 
process. 

The  concept  of  writing  a  verification  requirement  for 
each  performance  requirement  “shall  statement”  must 
be  incorporated  into  the  systems  engineering  process 
and  requirements  definition/specification  process 
including  software  engineering.  This  should  take  the 
form  of  a  narrative  section  four  in  the  specification. 
Examples  should  be  developed  that  would  illustrate 
how  to  write  verification  requirements.  Complete 
definition  of  verification  requirements  should  be  an 
integral  part  of  writing  the  specifications  and  be  part  of 
the  exit  criteria  for  the  formal  specification  review,  e.g., 
SRR  and  software  specification  review. 

This  one-to-one  performance-to-verification  narrative 
requirements  needs  to  be  emphasized  in  systems 
engineering  training  programs.  An  example  of  how  to 
white  these  in  the  JACG  series  of  specification  guides. 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 

Late  requirements  forced  into  software 

Apply  the  systems  engineering  requirements  analysis 
process  to  all  requirements  changes/additions 
throughout  the  development  cycle.  All 

changes/additions  should  be  formally  managed  through 
the  change  control  board  process.  Changes  would 
either  be  incorporated  through  an  individual  change 
request  or  a  “bucket”  ECP  (if  contract  change),  with 
adjustment  in  both  cost  and  schedule,  or  be  allocated 
to  a  planned  subsequent  contract  development 
increment.  The  decision  on  how  to  implement  the 
change  must  consider  the  criticality  and  magnitude  or 
the  change/additional  requirement.  Changes  that  do 
not  require  a  contract  change  should  also  go  through  a 
similar  contractor  controlled  change  process. 

Interface  requirements  and  Interface 
management 

Reinforce  the  importance  of  specifying,  allocating,  and 
controlling  interface  requirements  including  interfaces 
among  the  members  of  a  SoS/FoS.  In  particular, 
programs  need  to  include  verification  of  interface 
requirements  in  development  and  test  plans. 

Failure  to  implement  and  maintain 
requirements  traceability  tools 

Programs  anticipating  a  large  requirements  set  should 
institute  a  formal  requirements  traceability  process  that 
begins  with  the  ORD  and  evolves  to  include  the  total 
system. 

Require  contractors  to  implement  requirements  tracking 
tools  and  flow  down  these  tools  to  the  subcontractor 
base  to  assure  completeness.  When  appropriate,  use 
commercially  available  software  tools  to  implement  the 
requirements  management  and  tracking  process. 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 

Require  the  government  (operational  user  and  program 
office)  to  implement  a  disciplined  requirements 
management  process  from  the  ORD  through  the 
product  requirements  documents,  e.g.,  ORD  to  RFP 
performance  requirements  document  to  contract 
specifications  to  product  specifications,  etc. 

Systems  Engineering  Process 

SE  process  not  defined  on  program 

Programs  should  have  a  robust  systems  engineering 
process  that  is  comprehensively  defined  in  a  systems 
engineering  program  plan.  The  standard  corporate 
systems  engineering  process  can  be  applied  directly  to 
a  straightforward,  simple  program,  or  can  be  tailored  for 
a  more  complex/large/multi-contractor  team  program. 
The  proposal  solicitation  should  require.  In  section  L,  s 
systems  engineering  process  description  tailored  to  the 
program  as  part  of  the  proposal.  This  could  be  in  a 
contractor  format,  but  must  address  the  systems 
engineering  processes  as  they  will  be  applied  to  the 
program. 

SE  processes  defined  but  not  applied 

Map  the  systems  engineering  processes  defined  in  the 
program  systems  engineering  plan,  to  the  IMP, 
including  entrance  and  exit  criteria  for  the  program 
reviews  and  events.  Specifically  identify  systems 
engineering  task  completions  as  entrance  and  exit 
criteria.  Examine  planned  programmed  resources  to 
ascertain  if  adequate  resources  are  budgeted  to 
execute  the  proposed  systems  engineering  processes. 

Task  and  incentivize  contractors  to  follow  the  program’s 
systems  engineering  plan  through  the  use  of  award  fee. 
Include  metrics  that  measure  program  progress, 
processes  implemented,  and  are  tied  to  products. 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 

Train  the  program  participants  in  the  execution  of 
systems  engineering  processes,  including  value  of  the 
processes  and  importance  to  applying  the  systems 
engineering  processes. 

Lack  of  flow  down  of  SE  process 
requirements  to  development 

subcontractors 

Contractors  should  be  required  to  address  how  the 
systems  engineering  process  will  be  applied  across  all 
systems  and  subsystems  development  on  the  program 
in  their  proposal,  including  those  subsystems  that  are 
subcontracted.  This  systems  engineering  process  must 
be  described  in  the  proposal,  and  after  contract  award 
published  in  a  program  document  available  to  all 
program  participants  and  stakeholders.  Prime 
contractors  should  be  required  to  flow  down  a 
requirements  for  a  defined  written  systems  engineering 
process  to  their  development  subcontractors. 

Lack  of  robust  SE  applied  to  top  level 
system  design 

Require  completion  of  the  top  level  system  design  and 
requirements  allocation  to  subsystems  prior  to  initiation 
of  the  subsystem  design  activity.  Employ  IMP  criteria 
for  the  SRR  and  system  level  design  review  events  to 
assure  completeness. 

Include  basic  systems  engineering  on  programs  that 
have  substantial  requirements  definition  and  top  level 
efforts  prior  to  milestone  B,  e.g.,  during  technology 
development.  Complete  the  system  level  requirements 
and  design  per  defined  criteria  prior  to  MB  if 
appropriate. 

Lack  of  timely  system  integration  and  test 
(l&T)  planning 

Require  systems  level  integration  and  test  planning 
early  in  the  program.  Evaluate  the  IMP  and  IMS  for  test 
and  evaluation  planning  realism  during  the  source 
selection. 
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Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 

Require  all  test  requirements  be  identified  in 
specifications  by  the  SRR  and  plans  be  in  place  to 
satisfy  the  requirements  by  the  PDR;  include  test  and 
evaluation  as  a  topic  in  program  and  design  reviews. 
Assure  there  is  only  one  test  requirements  baseline  for 
the  program  that  all  stakeholders  use. 

Require  a  full  comprehensive  integration  and  test  plan 
as  entrance  criteria  for  CDR. 

Inadequate  planning  for  obsolescence  and 
sustainment 

Establish  requirements  early  in  the  program  to 
accomplish  technology  refresh  to  preclude 
obsolescence  early  in  the  program  and  include  the 
topic  throughout  the  development  process. 

Perform  the  expectancy  analyses  for  parts  susceptible 
to  early  obsolescence  as  part  of  the  systems 
engineering  processes  very  early  in  the  development 
cycle  and  include  design  considerations  that  facilitate 
technology  refresh. 

Include  sustainment  as  a  topic  throughout  the  design 
process;  include  sustainment  specialists  in  program 
planning  activities. 

Design  incorporated  into  performance 
specifications 

Employ  templates  for  performance  specifications; 
include  examples  such  as  those  contained  in  the 
JACG’s  series  of  specification  guides. 

Enforce  the  Performance  Based  Business  Environment 
(PBBE)  initiative  on  multipart  product  documentation 
that  separates  the  performance  language 

documentation  from  the  product  description  language 
documentation. 

Engineering  Management 
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Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 

Inadequate  schedules  and  budget  to 
accomplish  development  effort 

Senior  government  personnel  should  challenge 
unrealistic  schedules  at  Milestone  Reviews,  acquisition 
strategy  panels  (ASPs)  and  RFP  reviews.  For  software 
intensive  programs,  the  results  of  software  estimating 
models  runs  should  be  presented  to  these  reviews  to 
substantiate  the  viability  of  the  planned  schedule  and 
budgeted  resources.  In  addition,  planned  schedules 
should  consider  actual  schedules  from  past  completed 
programs. 

Sound  rationale  for  proposed  schedules  and  cost  tied 
to  performance  requirements  should  be  demanded  at 
Milestone  Decision  Authority  (MDA)  reviews. 

A  means  to  get  realistic  industry  feedback  based  on 
accurate  assessment  of  the  requirements  development 
effort,  rather  than  just  acceptance  of  risky  conditions 
stated  in  draft  proposal  solicitations,  needs  to  be 
established.  Industry  foes  challenge  performance 
requirements  on  occasion,  but  often  does  not  challenge 
immature  technology  expectations,  budget/cost 
baselines,  nor  schedule  expectations  for  fear  of  being 
non  responsive  or  not  competitive.  A  true  non¬ 
attribution  exchange  needs  to  occur. 

Assumed  reuse  not  confirmed 

Base  planned  reuse  and  COTS  on  factual  data.  Include 
systems  engineering  analyses  of  the  reuse  candidates 
to  determine  that  the  candidates  exist  in  acceptable 
usable  form,  that  they  need  meet  the  new  system 
requirements,  and  they  are  well  documented  and 
designed  for  reuse. 

Pan  ad  accomplish  early  demonstrations,  prior  to  PDR, 
of  the  reuse  candidates  on  system  simulations,  i.e. , 
verify  the  reusability  early.  Include  these 
demonstrations  in  the  risk  management  program  as 
risk  mitigation  activities. 
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Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 

Require  proposed  reuse  to  be  fully  supported  in  the 
proposals  and  consider  risks  in  the  evaluation.  The 
proposal  solicitation  should  include  a  reuse  checklist  to 
address  the  soundness  of  the  proposed  reuse 
candidates  and  approach,  confirm  reuse  compliance 
with  requirements,  quantify  the  attendant  risk,  and 
require  development  of  risk  mitigation  plans. 

Lack  of  active  engineering  management  of 
development  subcontractor 

RFPs  should  require  proposers  to  describe  the  systems 
engineering  processes  that  will  be  used  to  technically 
manage  subcontractors  development  key  subsystems. 

Contracts  with  the  primes  should  be  structured  to 
require  technical  engineering  management  visibility  and 
status  reporting  (metrics)  of  major  development 
activities  at  the  subcontractor  level  and  included  in  the 
prime  contractor’s  reporting  forward. 

Integrated  master  plan/integrated  master 
schedule  (IMSP/IMS)  not  adequately 
addressed  and/or  not  followed  on  the  program 

Require  the  offeror  to  propose  a  system  level  IMP  and 
IMS.  Evaluate  these  products  during  sources  selection 
and  make  sure  the  key  system  engineering  and 
engineering  d3evelopmetn  events  have  appropriate 
entry  level  criteria  and  the  adequacy  of  the  planned 
systems  engineering  task  durations. 

Keep  the  system  IMP/IMS  at  the  appropriate  level. 
Focus  on  key  gating  events  and  important  criteria. 
Establish  subsystem  IMP/IMS  as  appropriate  and  flow 
the  requirement  to  subcontractors  and  vendors  as 
appropriate. 

Incentivize  the  program  to  mange/execute  the  program 
following  the  IMP/IMS. 

Enforce  the  criteria  included  in  the  IMP;  don’t  just  gloss 
over  event  and  accomplishment  exit  criteria. 
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Best  Practice  or  Deficiency 

Recommendations 

Dayton  Aerospace  Quick  Study 
Report  on  the  Health  of  SE 
Process  Execution,  cont. 

When  actions  are  needed  to  close  criteria  partially  met, 
establish  near  term  follow-up  plans  to  assure  the 
actions  are  completed  and  the  IMP  criteria  are  fully 
met. 

Most  importantly  do  not  close  the  event,  e.g.,  CDR, 
until  all  established  exit  criteria  are  met. 

No  single  technical  focal  point 

Require  creation  of  a  single  technical  authority  that  is 
responsible  for  the  overall  technical  effort  and 
empowered  to  resolve  technical  issues.  This  position 
should  be  established  both  in  the  government  program 
office  and  the  contractor  program  office.  This  technical 
authority  should  report  to  the  Program  Director. 

Tri-Service  Assessment  Initiative 

No  defined  processes  in  place — Rudimentary 
processes  are  missing 

Program  team  processes  established,  but  not 
following  accepted  processes-Process 

adherence  shortfalls  in  requirements 

definition/management,  risk  management,  testing, 
and  systems  engineering  management 

Process  capability  should  be  evaluated  at  RFP  and 
major  milestone  reviews 

Poorly  executed  processes 

Constrained  processes 

Process  capability  shortfalls — program  team  is 
following  accepted  processes,  but  the  processes 
used  are  ineffective  for  the  specific  program 
situation  encountered 

Program  teams  must  be  educated  in  the  difference 
between  achieving  process  adherence  (following  some 
process)  and  implementing  process  capability  (the  true 
effectiveness  of  that  process) 

Outmoded  processes — process  model, 

standard  or  practice  no  longer  supported  or  is 
inappropriate  for  the  specific  program  situation 

Program  teams  need  to  evaluate  the  full  spectrum  of 
technical  and  managerial  process  requirements  and 
then  tailor  their  organizationally-based  adherence 
models  to  meet  specific  program  needs 
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Best  Practice  or  Deficiency 

Recommendations 

Tri-Service  Assessment 

Initiative,  cont. 

Pro  forma  processes — process  adequately 
defined  but  performed  in  a  “check  the  box” 
manner 

Individual  program  team  members  need  to  ensure 
collectively  that  their  technical  and  managerial 
processes  meet  the  needs  of  the  program  team  as  a 
whole.  Too  often,  program  team  members  simply 
adhere  to  their  process  model,  standard,  or  individual 
best  practice,  without  understanding  how  their  process 
may  clash  with  another  team  member’s  processes,  or 
whether  it  is  adequate  for  the  specific  program  in 
question. 

Non-integrated  team  processes — full  spectrum 
of  program  team’s  organizational  processes 
are  not  rigorously  evaluated  and  then  tailored 
to  meet  the  specific  characteristics  or 
requirements  of  the  program 

DoD  must  foster  innovative  processes  and  practices 
that  are  capable  of  handling  the  emerging  complexity  of 
system  acquisitions,  developments,  and  deployments 

No  accepted  processes  defined-Emerging 
processes/missing  innovative  processes — largely 
revised  or  new  process  is  required  but  program 
team  has  failed  to  define  it  in  sufficient  detail  so 
that  the  process  adequately  addresses  the 
identified  risk  or  problem 

Space  Broad  Area  Review 

System  design  and  process  engineering 
deficiencies  played  a  prominent  role  in  failures 
and  near  misses  --  program  management 

Failures  and  major  anomalies  reflect  the  need  for 
contractor  program  management  to  provide  more 
disciplined  systems  engineering  in  design  and 
processes 
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Best  Practice  or  Deficiency 

Recommendations 

Space  Broad  Area  Review,  cont. 

Formal  risk  management  policies,  practices  and 
procedures  have  been  degraded  over  the  past 
decade — No  formal  technical  risk  management 
process 

Institutionalize  a  formal  launch  risk  management 
program 

-  Develop  and  manage  a  risk  management  plan  for  all 
fly-out  systems 

-  Emphasize  identifying  and  mitigating  risks 

-  Formalize  systems  engineering  and  quality  policies, 
practices  and  procedures 

-  Re-institute  a  comprehensive  post-flight  analysis 
program 

Engineering  process  and  workmanship  are  both 
prominent  in  mishaps  since  1985  --  historically 
consistent  themes 

-  Lack  of  disciplined  system  engineering  in  design 
and  processing  of  launch  vehicles 

-  Lack  of  communication  of  critical  data 

-  Inadequately  defined  processes 

-  Inadequate  review  process  --  particularly  in 
design  and  design  change 

Air  Force  formulate  a  formal  EELV  launch  risk 
management  program 

-  Develop  and  manage  a  risk  management  plan  for 
EELV  systems 

-  Formalize  systems  engineering  and  quality  policies, 
practices  and  procedures 

-  Develop  and  implement  an  improved  mission 
assurance  process  based  on  the  best  attributes  of 
SMC,  NASA  and  NRO  mission  assurance  practices 

NDIA  Top  Five  SE  Issues 

Lack  of  awareness  of  the  importance,  value, 
timing,  accountability,  and  organizational  structure 
of  SE  on  programs 

Increase  awareness  of  SE  importance  within 
acquisition  formulation  and  decision  processes  early 
and  consistently  over  major  milestones,  and  recognize 
SE  authority  and  responsibility  in  the  ACAT  IC/D 
process  present  during  the  acquisition  formulation  and 
decision  processes,  with  similar  efforts  at  lower 
program  levels. 

General  awareness  of  cost  benefit  and  return 
on  investment  performance  of  SE  possesses  is 
unknown  It  would  be  most  helpful  to  have  a 
“lessons  learned”  repository 

SE  is  NOT  an  option.  It  is  an  essential  ingredient  on 
all  programs  and  must  have  adequate  funding. 
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NDIA  Top  Five  SE  Issues,  cont. 

There  is  a  lack  of  commitment  to  SE  across  the 
life  cycle  of  programs 

PMs  are  accountable  and  responsible  for  SE 
implementation  across  entire  life  cycle 

There  is  a  lack  of  understanding  on  the 
application  of  SE  principles,  content  and  areas 
of  application  to  program  success,  including 
areas  such  as  risk  management,  security, 
COTS  integration,  architectures,  identification 
of  cost  drivers,  test  &  evaluation,  and 
supportability/life  cycle  cost 

It  would  be  most  helpful  to  have  a  “lessons  learned” 
repository 

Program  managers,  both  industry  and 
government,  do  not  have  adequate  recognition 
that  SE  touches  all  aspects  of  the  acquisition 
process,  e.g.  from  up-front  requirements, 
budgeting,  to  end  of  life  disposition 

Systems  Engineering  content,  when  included  in 
proposal  costs  as  a  factor  versus  bottom-up 
quoting,  exacerbates  the  problem  of  funding 
when  costs  are  being  evaluated  as  it  is  looked 
at  as  “overhead”  that  becomes  a  prime 
candidate  for  cost  cutting 

Adequate,  qualified  resources  are  generally  not 
available  within  Government  and  industry  for 
allocation  on  major  program 

Establish  a  program  and  process  for  incentivizing 
career  systems  engineer  positions  within  the 
Government. 

An  experienced,  trained  workforce  is  in  short 
supply.  There  is  sufficient  opportunity  for 
systems  engineers  in  Government  but 
inequities  in 

compensation  and  incentive  versus  industry  for 
systems  engineers  at  midcareer  (e.g.  10-20 
years)  encourages  migration  out  of 
Government  service. 

Require  the  SEMP  to  identify  the  process  and  qualification 
requirements  for  all  key  personnel  proposed  for  the 
contract 
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NDIA  Top  Five  SE  Issues,  cont. 

We  do  not  consistently  allocate  adequate, 
trained  resources  on  programs 

Work  with  major  universities  to  require  an  introductory 
course  in  systems  engineering  in  all  undergraduate  and 
graduate  level  engineering  and  technical  management 
degree  programs. 

Program  management  turnover  is  an 
impediment  to  success 

Require  that  program  managers  receive  systems 
engineering  training  so  they  understand  the  significance 
that  SE  plays  in  assuring  program  success 

The  DAWIA  certification  process  levies  only 
minimal  academic  requirements  and  does  not 
produce  adequately  trained  systems  engineers 
because  of  inadequate  systems  engineering 
experience  is  required  as  a  qualification 

Ensure  that  DAU  SE  level  2  and  3  courses  address,  as  a 
minimum,  training  on  the  SE  tools  processes  and 
documents  as  defined  in  SAF  AQ  memo  dated  January 
06,  2003,  "Incentivizing  Contractors  for  Better  Systems 
Engineering". 

Opportunities  in  industry  to  gain  systems 
engineering  experience  are  limited  by  the 
number  of  and  completeness  of  systems 
engineering  content  on  programs 

A  limited  number  of  academic  and  practical 
sources  of  systems  engineering  curricula  and  a 
very  limited  number  of  graduates  are  available 
to  either  industry  or  government 

There  are  inadequate  certification  methods  in 
both  industry  and  government  to  insure  the 
quality  and  competency  of  systems  engineer 

Insufficient  SE  tools  and  environments  to 
effectively  execute  SE  on  programs 

Research  and  identify  SE  tools  for  system  architecture 
design  and  development,  and  encourage  use  thereof. 

The  SE  community  lacks  a  set  of 
comprehensive,  common  and  consistent  tools, 
guidance  and  standards,  and  metrics,  which 
leads  to  stovepipes  and  inadequate  data  and 
data  transfer 

There  is  need  for  development  of  a  system 
architecture  framework  for  particular  system  in 
accordance  with  FEW  [federal  enterprise 
architecture/CBA-component  based  architecture,  the 
DoD-directed  Zachman  framework,  and  C4ISR 
three-schema  architecture] 
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NDIA  Top  Five  SE  Issues,  cont. 


In  terms  of  the  environment  definition  -  there  is 
insufficient  understanding  of  both  the  individual 
and  System-of-System  interdependency  of 
product,  process,  and  people/organization 
environments.  There  are  insufficient  SE 


Efforts  should  identify  all  of  the  program-applicable 
domains  (government,  primes  and  major 
subcontractors)  and  environments  required  for  a 
structured  hierarchical  decomposition  of  a  weapon 
or  IT  system 


tools  to  properly  define  the  various 
environments  (Modeling  and  Simulation 
domain.) 

Substantial  data  inconsistencies  exist  -  there  is 
no  common  data  dictionary,  use  of  metadata 
(meaning  and  definition  of  data),  Configuration 
Management  of  data,  and  use  of  common 
terminology.  There  is  insufficient 
Validation/Verification  and  certification  of  data, 
resulting  in  little  data  sharing  and  reuse 

In  terms  of  tools  and  product  environments, 
there  is  no  overall  SE  tool  system,  from  a 
hierarchical  standpoint,  that  incorporates  the 
detail  design,  life  cycle  cost,  and  overall 
performance,  including  those  needed  for  SOS 
and  Interoperability. 

There  is  little  or  no  integration  of  Systems 
Engineering  Computer  Aided  Engineering 
(CAE)  tools  with  software  and  hardware  design 
tools,  and  likewise  between  the  tools  and 
models  used  by  government  and  Industry 

There  is  no  requirement  to  use  a  requirements 
management  s/w  tool  (such  as  DOORS,  RTM, 
or  SLATE) 
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NDIA  Top  Five  SE  Issues,  cont. 

Requirements  definition,  development  and 
management  is  not  applied  consistently  and 
effectively 

Synchronize  directives  used  by  the  acquisition  and 
requirements  community  to  ensure  a  disciplined  and 
consistent  requirements  definition  and  development 
approach. 

The  Government,  for  the  most  part,  within  the 
contracting  and  war  fighting/operational 
communities  do  not  have  the  Systems 
Engineering  mentality  in  addressing  Mission 
Needs  and  requirements  and  do  not  follow  their 
requirements  process  effectively. 

The  education  process,  both  formal  and  informal,  for 
Government  Program  Managers  and  contractors 
should  be  sufficient  that  they  mutually  understand 
the  necessity  of  a  comprehensive  architectural 
approach  and  systems  engineering  focus  in  applying 
the  complete  and  managed  requirements  process  on 
programs 

Acquisition  and  requirements  communities 
follow  different  directives 

OSD  should  link  requirements  definition,  development 
and  management  into  the  program  life  cycle  through 
defined  practice  and  guidance 

There  is  often  a  lack  of  understanding  by 
contractors  /  government  of  true  capabilities 
and  requirements  needed  by  the  war-fighter, 
resulting  in  incomplete  or  inaccurate 
Requirements  Definition  with  respect  to 
implications  of  stated  requirements 

Emphasize  process  maturity  related  to  requirements  for 
both  acquisition  and  development  communities  by 
adoption  of  maturity  models  such  as  CMMI 
(Capability  Maturity  Model  Integration) 

There  is  a  serious  lack  of  upfront  and 
continuous  requirements  development  and 
management,  including  management  of 
requirements  changes 

Involve  potential  contractors  in  the  requirements 
definition  process  early  in  the  acquisition  cycle 

Determining  adequate  and  correct 

requirements  for  software  intensive  systems 
that  satisfy  the  overall  systems  objectives  is  an 
elusive  task 
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NDIA  Top  Five  SE  Issues,  cont. 

We  do  not  adequately  plan  for  systems 
adaptability/reconfigurability  to  support  rapidly 
changing  requirements,  both  customer  and 
end-user,  (i.e.,  spiral  development  and 
evolutionary  acquisition)  and  technology 
insertion  requirements  not  generally  not 
defined  at  program  outset 

Poor  initial  program  formulation  put  successful 
execution  at  risk.  Initial  Program  Formulation 
begins  prior  to  at  Milestone  0  and  culminates  with 
Award  of  SDD  Contract.  During  this  timeframe, 
many  critical  decisions  are  made  that  have 
potentially  profound  impact  on  the  program. 
These  include  establishing  initial  cost  and 
schedule  baselines,  system  requirements,  and  the 
management  approach. 

Emphasize  the  use  of  architecture  development  and 
systems  engineering  practice  in  the  initial  program 
formulation  phase  including  the  use  of  realistic 
estimates  for  cost  and  schedule,  risk  identification,  and 
clearly  defining  requirements.  Even  if  the  program  does 
not  include  all  life  cycle  phases,  consideration  of 
supportability  needs  to  be  included  from  the  outset. 

Government  &  Contractor  Program  Managers 
are  required  to  meet  predefined  baseline  cost 
&  schedule.  Early  estimates  of  program  cost 
and  schedule  need  to  be  made  based  upon  a 
true  and  valid  estimate  of  the  work,  not  the 
desired  price  and  timeline.  This  is  at  odds  with 
the  environment  under  an  evolutionary 
acquisition  approach. 

Modify  the  current  acquisition  approach  to 
encourage  more  candid  communication  of  program 
cost,  schedule  and  risk  between  Government  and 
Industry. 

Moreover,  if  the  attempt  is  made  to  make 
programs  more  palatable  by  using  optimistic 
scheduling  and  cost  estimating  initially,  the 
program’s  ability  to  establish  and  consistently 
implement  effective  systems  engineering  and 
associated  processes  is  immediately  put  at  risk 

Encourage  that  initial  engineering  go  beyond  the 
superficial,  either  through  pre-acquisition  activities 
and  funded  studies  and  analysis  so  that  initial 
program  formulation  accurately  predicts  schedule, 
cost  and  risk. 
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Source 

Best  Practice  or  Deficiency 

Recommendations 

NDIA  Top  Five  SE  Issues,  cont. 

When  the  initial  funding  profile  does  not 
address  full  lifecycle,  later  attempts  to 
recognize  the  true  implications  of  the  initial 
program  could  result  in  unfunded  surprises  and 
the  eventual  down  sizing  of  the  initial  program. 

Ensure  that  unrealistic  or  incompatible  cost, 
schedule,  and  performance  baselines  are  clearly 
identified  as  a  risk,  so  that  the  situation  can  be 

effectively  managed. 

The  guidelines  used  for  acquisition  are 
sometimes  interpreted  as  discouraging  the 
telling  the  real  story  under  the  guise  of 
ensuring  competition  or  “keeping  the  program 
alive”.  Clearly  exposing  the  requirements, 
constraints,  ramifications  and  implications  of 
variants,  criteria  for  decision  making,  budgetary 
issues,  and  customer  desires  would  lead  to 
better  understanding  on  the  part  of  the 
acquisition  staff,  the  PMO,  and  the  bidding 
contractors  and  to  successful  program  results. 

Emphasize  investigation  of  the  implications  of  initial 
requirements  statements,  not  only  on  design  but 
also  on  supportability. 

Program  organizational  structures,  both 
government  and  industry,  tend  to  enable  IPTs 
too  early  in  the  execution  phase,  before  system 
level  requirements  are  firm  and  able  to  be 
allocated  to  the  element/subsystem  level.  Such 
strong  IPTs  and  weak  Systems  Engineering 
Integrated  Teams  (SEITs)  or  Overarching  IPTs 
(OIPTs)  foster  integration  problems  at  the 
system  level. 

Encourage  early  and  strong  government/industry 
SEIT  or  OIPT  activity  prior  to  formation  and 
chartering  of  specific  IPTs 

There  is  inadequate  emphasis  on 

architecturally  scalable  designs  and 

development  strategies  that  can  readily 
accommodate  normal  requirements  growth, 
especially  those  programs  involving  spiral 
development  and  evolutionary  acquisition. 

Emphasize  process  maturity  in  contractor 
communities,  applicable  to  all  phases  of  a  program, 
by  adoption  of  maturity  models  such  as  CMMI 

(Capability  Maturity  Model  Integration) 
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DAU’s  Systems  Engineering  Department 
Revamping  SYS-301  Course 

Systems  Engineering  Competencies  at 
Core  of  Recent  Changes 
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Systems  engineering  is  an  inter¬ 
disciplinary  engineering  man¬ 
agement  process  that  evolves  and 
verifies  an  integrated,  life  cycle 
balanced  set  of  system  solutions 
that  satisfies  customer  needs.  It  clearly 
is  at  the  heart  of  the  systems  acquisition 
process,  and  the  DoD  relies  heavily  on 
systems  engineers  to  provide  technical 
support  to  program  managers.  In  fact, 
the  Systems  Planning,  Research,  Devel¬ 
opment  and  Engineering  (SPRDE)  ca¬ 
reer  field  has  more  members  than  any 
of  the  other  12  Department  of  Defense 
(DoD)  acquisition  career  fields.  One  way 
the  DoD  ensures  that  its  systems  engi¬ 
neers  possess  the  needed  competencies 
to  perform  their  jobs  is  through  a  certi¬ 
fication  process  that  includes  specific 
training  requirements. 

The  Original  SYS-301  Course 

In  1991  Congress  passed  the  Defense 
Acquisition  Workforce  Improvement 
Act  (DAWIA).  In  response,  DoD  created 
the  Acquisition  Workforce  Certification 
Program.  This  program  established  ed¬ 
ucation,  training,  and  experience  crite¬ 
ria  for  each  of  the  13  acquisition  career 
fields.  The  SPRDE  career  field  has  as  one 
of  its  criteria  for  level  III  certification 
completion  of  a  two-week  course  called 
Systems  301  (SYS-301),  Advanced  Sys¬ 
tems  Planning,  Research,  Development 
and  Engineering  (ASPRDEC). 

As  the  first  step  in  developing  this 
course,  the  Defense  Acquisition  Uni¬ 
versity  (DAU)  conducted  a  number  of 
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workshops  with  field  activities  to  get 
their  input  on  what  material  should  be 
covered.  We  received  over  400  distinct 
suggestions,  ranging  from  broad  areas 
such  as  configuration  management  to 
specific  skills  such  as  being  able  to  do 
word  processing.  These  inputs  were  nar¬ 
rowed  down  into  about  30  topic  areas 
by  combining  related  items.  Lesson 
plans,  including  learning  objectives, 
were  then  developed  for  each  topic  area. 
A  DoD- level  functional  board  oversaw 
this  process  and  by  1994  the  first  class 
was  held. 

Need  for  Change 

In  subsequent  years,  course  materials 
were  updated  to  ensure  currency  and 
minor  changes  were  made  to  most 


lessons  to  incorporate  suggested  im¬ 
provements,  but  the  learning  objectives 
and  course  structure  remained  relatively 
stable.  By  the  end  of  1999,  however,  it 
was  becoming  obvious  that  the  skills 
and  competencies  needed  by  senior 
technical  managers  were  changing.  Is¬ 
sues  such  as  the  need  for  systems  in¬ 
teroperability  and  increasing  use  of  soft¬ 
ware  to  perform  system  functions  were 
becoming  vitally  important.  On  top  of 
this,  a  major  change  in  how  the  DoD 
would  do  systems  acquisition  was  soon 
to  be  expressed  in  the  new  5000  series 
of  acquisition  regulations. 

Recognizing  that  it  was  time  to  make  a 
substantial  revision  to  the  SYS-301 
course,  DAlte  Systems  Engineering  De¬ 


partment  began  a  process  of  interview¬ 
ing  20  program  managers  and  techni¬ 
cal  managers  from  the  three  Services, 
other  agencies,  and  industry  Managers 
were  asked  what  they  felt  were  the  skills 
that  senior  technical  managers  needed 
to  be  most  effective.  These  skills  were 
then  consolidated  into  15  skill  areas. 

Once  these  skill  areas  were  compared 
to  the  SYS-301  course  content,  we  dis¬ 
covered  that  several  subjects  covered  in 
the  course — such  as  International  Ac¬ 
quisition  and  Environment,  Safety,  and 
Health  (ESH) — had  not  been  mentioned 
in  the  interviews.  Since  the  goal  of  this 
research  was  to  provide  direction  for  a 
cumculum  revision,  we  decided  to  mod¬ 
ify  the  original  15  areas  to  ensure  that 
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FIGURE  1  Systems  Engineer  Competencies 


1.  Total  Systems  View 

Ability  to  think  beyond  engneering  and  con¬ 
sider  all  functions  and  stakeholders  in  the 
Systems  Engineering  process  Ability  to 
understand  the  entire  acquisition  process 

2  Teaming 

Ability  to  build,  work  in.  motivate,  and  lead 
high-performing  multidisciplinary  teams. 

3.  User  Focus 

Ability  to  understand  the  user's  perspective 
and  requirements.  Ability  to  conduct  require¬ 
ments  analysis  and  to  structure  Research  and 
Development  work  effort  to  match  user 
needs. 

4.  contract  Technical  Management 

Ability  to  understand  contractors'  processes 
and  perspectives,  to  work  with  contractors 
and  provide  informed  assessments  of  their 
progress,  and  to  understand  the  source 
selection  process. 

5.  Configuration  Management 

Ability  to  manage  and  communicate  changes 
to  systems  in  all  phases  of  the  life  cycle. 

6.  Post  Production  Support 

Ability  to  identify  improvements  to  systems 
for  the  purpose  of  Operations  and  Support 
(O&S)  cost  reduction,  safety,  replacing  obso¬ 
lete  parts,  reliability,  tech  insertion,  etc.  Ability 
to  use  Systems  Engineering  process  to  imple¬ 
ment  these  changes. 

7.  Financial  Management 

Ability  to  understand  the  Planning.  Program¬ 
ming  and  Budgeting  System  (PPBS)  system, 
sources  and  uses  of  funds,  and  how  budget 
issues  impact  the  program. 

8.  Operational  Cost  Reduction 

Ability  to  assess  design  impact  on  Total  Oper¬ 
ational  Cost  and  identify  means  to  reduce 


Total  Ownership  Cost  (TOC).  Ability  to  under¬ 
stand  designing  for  change  using  techniques 
such  as  open  systems  architectures  Ability  to 
understand  Cost  As  an  independent  Variable 
(CAIV). 

9  Risk  Management 

Ability  to  plan,  assess,  handle,  and  monitor 
risk. 

10.  Management  of  Changing  Tech¬ 
nology 

Awareness  of  current  state-of-the-art.  and 
mechanisms  to  introduce  technology.  Ability 
to  accurately  assess  technoloa  maturity  of  a 
system.  Ability  to  understand  Information 
Technology  and  how  to  effectively  acquire 
and  use  it.  Ability  to  understand  spectrum 
management,  system  security,  and  Joint 
Technical  Architecture. 

11  Earned  Value 

Ability  to  understand  Earned  Value  ( EV)  prin¬ 
ciples.  evaluate  EV  data,  and  make 
recommendations. 

12.  Software  Management 

Knowledge  of  software  development  princi¬ 
ples  and  techniques.  Ability  to  integrate  soft¬ 
ware  considerations  into  the  systems 
engineering  process.  Ability  to  assess  devel¬ 
opment  progress  and  identify  risks  and 
pitfalls 

13.  Design  Impacts  on  Manufacturing 

Understanding  of  producibility  issues  and 
hew  to  manage  the  design  for  producilility. 

14  Modeling  and  Simulation 
Ability  to  understand  uses  of  Modeling  and 
Simulation  (M&S).  Ability  to  use  M&S 
throughout  the  life  cycle  and  to  assess  the 
contractor's  use  of  M&S.  Ability  to  work  in  an 
integrated  data  environment 


15.  Ethics 

Ability  to  understand  ethical  considerations 
and  adhere  to  ethical  principles. 

16  Environment,  Safety  and  Health 
(ESH) 

Ability  to  understand  Environment  Safety 
and  Health  (ESH)  requirements  and  how  to 
design  systems  to  effectively  meet  those  re¬ 
quirements. 

17  International  Acquisition  (IA) 

Ability  to  understand  International  Acquisition 
(IA)  policy  and  techniques.  Ability  to  utilize 
offshore  technology  in  system  design  where  it 
provides  a  benefit  Ability  to  understand  inter¬ 
operability  requirements. 

18  Test  Integration 

Ability  to  assist  in  test  planning  and  design 
Ability  to  respond  to  issues  arising  during  test. 

19.  Logistics  Integration 

Ability  to  understand  designing  for  supporta- 
bility.  Ability  to  develop  or  evaluate  design 
changes  In  response  to  supportability  issues. 

20.  Evolutionary  Acquisition/Open  Sys¬ 
tems  Architecture 

Ability  to  develop  Evolutionary  Acquisition 
(EAl  desigi  strategies  and  ensure  system  de¬ 
sign  supports  the  EA  approach  Ability  to  un¬ 
derstand  open  systems  architectures. 

21 .  Assimilation  and  Communication  of 
Technical  Information 

Ability  to  evaluate  technical  issues,  assess 
program  performance,  make  recommenda¬ 
tions.  and  effectively  present  these  issues  to 
diverse  audiences 

22.  Adaptability 

Ability  to  respond  quickly  and  effectively  to 
changing  conditions  or  events  that  impact 
the  program's  systems  engineering  process. 
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all  topics  already  covered  in  the  course 
were  included. 

Figure  1  displays  the  final  list  of  22  areas. 
We  termed  these  “Systems  Engineer 
Competencies.”  By  no  means  is  this  a 
comprehensive  list  of  all  competencies 
required  by  systems  engineers.  Domain- 
specific  technical  knowledge,  organiza¬ 
tional  knowledge,  management  skills, 
and  many  behavioral  competencies  are 
not  included.  These  competencies  were 
beyond  the  scope  of  SYS-301 ,  and  it  was 
assumed  they  would  be  acquired 
through  other  means. 

The  interview  process  did  not  prioritize 
the  competencies  and  used  too  small  a 
sample  to  treat  it  as  representing  the 
thoughts  of  the  entire  acquisition  com¬ 
munity.  The  next  step,  therefore,  was  to 
develop  a  questionnaire  that  could  be 
administered  to  a  broad  cross-section  of 
senior  program  managers,  engineers, 
and  technical  managers. 

The  first  part  of  the  questionnaire  asked 
a  number  of  demographic  questions. 
The  second  part,  displayed  in  Figure 
2,  asked  respondents  to  rate  each  of 
the  22  competencies  as  having  high, 
medium,  or  low  importance.  In  order 
to  distinguish  those  of  highest  im¬ 
portance,  respondents  were  asked  to 
rate  not  more  than  eight  competen¬ 
cies  as  high. 

The  questionnaire  also  asked  for  an  as¬ 
sessment  of  the  degree  to  which  DoD^ 
technical  workforce  possessed  each 
competency.  We  did  this  to  allow  an 
analysis  showing  where  there  were  gaps 
between:  1)  how*  important  a  compe¬ 
tency'  is,  and  2)  the  current  level  of  com¬ 
petence.  Space  was  provided  for  re¬ 
spondents  to  add  any  competencies  not 
on  the  list  that  they  thought  were  im¬ 
portant. 

The  survey  was  administered  to  137 
students  while  they  attended  the  SYS- 
301  class,  to  96  students  in  the  Ad¬ 
vanced  Program  Management  Course 
(APMC),  and  to  90  senior  technical 
managers  throughout  DoD.  Respon¬ 
dents  were  able  to  fill  out  the  question¬ 
naire  online  with  their  input  going  di- 
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rectly  to  the  Center  for  Research,  De¬ 
fense  Acquisition  University.  They  were 
then  able  to  analyze  and  sort  responses 
by  Service,  years  of  experience,  rank, 
and  several  other  categories  using  the 
“GroupSystems  Survey  Tool”  by  Group- 
Systems.com 

Study  Results 

Figure  2  shows  the  overall  rankings  for 
the  entire  population  of  323  responses, 
with  competencies  listed  in  descending 
order  of  importance.  The  correspond¬ 
ing  assessment  of  the  degree  to  which 
people  in  the  SPRDE  career  field  pos¬ 
sess  these  competencies  is  listed  in  the 
right  column  of  Figure  2. 

The  first  four  competencies  were  rated 
“high”  importance  by  the  majority  of 
the  respondents.  The  next  nine  had  at 
least  twice  as  many  “high”  rating?  as 
“low”  rating?.  Only  the  last  three  had 
more  “low”  than  “high”  rating?.  Of  in¬ 
terest  is  the  frequency  of  “low"  ratings 
for  the  observed  degree  of  competence. 
Only  five  of  the  22  competencies  re¬ 
ceived  more  “high”  than  “low”  rating. 


Half  had  more  than  twice  as  many  “low” 
ratings  as  “high"  ratings. 

For  the  most  part,  the  three  sample 
groups  had  similar  rankings  for  both 
“importance"  and  “degree  observed.” 
There  were,  however,  some  notable  ex¬ 
ceptions. 

•  The  Advanced  Program  Management 
Course  students  ranked  “operational 
cost  reduction"  and  “financial  man¬ 
agement”  much  higher  in  importance 
and  “test  integration"  much  lower  in 
importance. 

•  Senior  technical  managers  ranked 
“ethics”  much  higher  in  importance. 

•  The  ASPRDEC  students  ranked  “mod¬ 
eling  and  simulation”  much  higher 
and  “adaptability”  much  lower  in  im¬ 
portance. 

•  The  ASPRDEC  students  ranked  “as¬ 
similation  and  communication  of 
technical  information”  and  “adapt¬ 
ability”  much  lower  for  degree  ob¬ 
served. 

•  Senior  managers  ranked  “user  focus” 
much  lower  for  degree  observed. 


FIGURE  2.  Questionnaire 

Competed  cy 

Importance 

Observed 

1.  Total  Systems  View 

High 

Moderate 

2.  User  Focus 

High 

Moderate 

3.  Risk  Management 

High 

Moderate 

4.  Teaming 

High 

Moderate 

5.  Assimilation  &  Communication  of 

Moderate-High 

Moderate 

Technical  Information 

6.  Software  Management 

Moderate-H^h 

Moderate-Low 

7.  Management  of  Changing  Technology 

Moderate-High 

Moderate-Low 

8.  Test  Integration 

Moderate-High 

Moderate 

9.  Operational  Cost  Reduction 

Moderate-High 

Moderate-Low 

10.  Adaptability 

Moderate-H^h 

Moderate-Low 

11 .  Logistics  Integration 

Moderate-High 

Moderate 

12.  Configuration  Management 

Moderate-High 

Moderate 

13.  Contract  Technical  Management 

Moderate-High 

Moderate 

14.  Evolutionary  AcquisitioiYOpen 

Moderate 

Moderate-Low 

Systems  Architecture 

15.  Financial  Management 

Moderate 

Moderate-Low 

16.  Desgn  Impacts  on  Manufacturing 

Moderate 

Moderate-Low 

17.  Ethics 

Moderate 

Moderate-High 

18.  Modeling  &  Simulation 

Moderate 

Moderate-Low 

19.  Post  Production  Support 

Moderate 

Low 

20.  Earned  Value 

Moderate- Low 

Moderate-Low 

21 .  Environment,  Safety  &  Health 

Moderate-Low 

Moderate 

22.  International  Acquisition 

Moderate- Low 

Low 
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Since  the  three  groups  had  different  per¬ 
spectives  due  to  the  nature  of  their  jobs, 
some  differences  were  to  be  expected. 

Analysis 

Our  next  step  was  to  determine  what 
this  research  suggested  as  to  how  we 
should  adjust  the  SYS-301  course  ma¬ 
terial.  Rather  than  merely  focus  on  the 
most  important  competencies,  we  felt 
it  important  to  take  into  account  how 
well  the  SPRDE  population  was  already 
doing  in  each  area.  The  thought  here 
was  that  areas  where  we  are  already 
doing  well  may  not  need  extra  empha¬ 
sis,  even  if  they  are  important.  Con¬ 
versely,  areas  where  we  are  doing  poorly 
may  need  extra  emphasis,  even  if  they 
are  not  the  most  important. 

To  help  us  in  this  assessment,  we  did  a 
“gap  analysis.”  We  assigned  numerical 
values  to  high  (8)%  medium  (4),  and  low 
(1)  ratings  and  then  averaged  the  re¬ 
sponses  to  get  numerical  values  for  each 
competency.  The  results  (Figure  3,  next 
page)  show  that  significant  gaps  exist  in 
about  half  of  the  competencies. 

After  evaluating  this  information  we 
drew  the  following  conclusions  about 
how  much  emphasis  each  area  should 
receive  in  SYS-301.  This  does  not  imply 
that  any  of  these  areas  are  unimportant. 
It  just  recognizes  that  time  constraints 
limit  w'hat  material  can  be  covered  in 
one  class.  To  the  extent  students  need 
additional  training  in  areas  we  don’t 
strongly  emphasize,  other  resources  are 
available  to  provide  that  training. 

Strongly  emphasize 
Total  Systems  View  (H) 

Risk  Management  (H) 

Teaming  (H) 

User  Focus  (M) 

Assimilation  and  Communication  of 
Technical  Infonnation  (M) 

Software  Management  (L) 

Management  of  Changing  Technology7 
(M) 

Emphasize 

Operational  Cost  Reduction  (L) 
Adaptability  (L) 

Evolutionary*  Acquisition/Open  Systems 
(M) 
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Logistics  Integration  (L) 

Test  Integration  (L) 

Emphasize  somewhat 
Modeling  and  Simulation  (M) 

Post  Production  Support  (L) 

Design  for  Manufacturing  (M) 
Financial  Management  (L) 

Contract  Technical  Management  (M) 
Configuration  Mmanagement  (L) 

Little  emphasis 
Earned  Value  (L) 

Ethics  (M) 

Environment,  Safety  and  Health  (M) 
International  Acquisition  (M) 

The  next  step  w'as  to  determine  how 
much  emphasis  each  area  w'as  already 
receiving  in  SYS-301.  All  faculty7  mem¬ 
bers  then  teaching  the  class  were  asked 
to  estimate  how  many  hours  were  spent 
in  each  area.  Areas  with  more  than  4 
hours  were  ranked  "high,”  those  with 
between  2  and  4  hours  w*ere  ranked 
"medium,”  and  areas  with  less  than  2 
hours  were  ranked  “low7.”  These  ratings 
are  shown  in  parentheses  in  the  areas  of 
emphasis  covered  above. 


Students  in  three  ASPRDEC  classes  were 
also  asked  their  assessments  of  wiiat  w'as 
actually  being  taught.  They  agreed 
closely  with  the  instructors  except  for 
the  area  “management  of  changing  tech¬ 
nology,”  which  they  felt  should  have 
much  more  emphasis.  A  comparison 
between  what  was  needed  and  what  was 
being  taught  show's  agreement  in  most 
areas,  but  it  also  highlighted  several  areas 
where  changes  were  indicated. 


More  Emphasis 

Software  Management 
Operational  Cost  Reduction 
Management  of  Changing  Technology 

Less  Emphasis 

International 

Environment,  Safety  and  Health 
Ethics 


DoD  Workforce  Report 

During  this  same  period,  the  Office  of 
the  Under  Secretary  of  Defense  for  Ac¬ 
quisition,  Technology7  and  Logistics  was 
conducting  a  study  of  compe  tencies  re¬ 
quired  by  the  DoD  acquisition  work- 
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force.  This  study  resulted  in  a  report  ti¬ 
tled  “Future  Acquisition  and  Technol¬ 
ogy  Workforce  Report."  This  report 
listed  435  functional  competencies.  Of 
these,  274  were  relevant  to  the  SPRDE 
career  field.  These  were  grouped  into 
33  “environmental  trends." 

A  comparison  with  our  study  results  in¬ 
dicated  very  good  agreement.  For  ex¬ 
ample,  rwo  of  the  environmental  trends 
were  “increased  emphasis  on  interop¬ 
erability"  and  "increased  emphasis  on 
software  development."  A  number  of 
environmental  areas  addressed  the  need 
to  reduce  operational  costs  through  a 
variety  of  means. 

Implementation 

With  this  information  in  hand,  w'e  began 
to  incorporate  changes  into  the  course 
material.  A  new  two-hour  lesson  titled 
“Architecture  and  Interoperability"  ad¬ 
dressed  systems  architectures  and  cur¬ 
rent  policy  on  interoperability  require¬ 
ments  and  certification.  This  knowledge 
is  then  used  in  a  case  study  where  stu¬ 
dents  develop  a  system  architecture  and 
look  at  interface  requirements.  Six  hours 
of  instruction  in  software  acquisition 
were  added.  Topics  covered  include  pol¬ 
icy,  development  strategies,  the  software 
life  cycle,  best  practices,  and  software 
risk  management. 

In  order  to  make  space  for  this  new  ma- 
tenal,  we  decided  to  eliminate  the  Con¬ 
tracting  Issues  lesson  and  shorten  the 
ESH  and  International  Acquisition 


lessons.  While  the  survey  results  didn’t 
make  a  strong  case  to  eliminate  the  con¬ 
tracting  lesson,  we  felt  that  most  of  the 
students  had  already  received  more  con¬ 
tracting  training  than  we  were  provid¬ 
ing.  We  also  felt  that  those  students 
needing  more  training  in  this  area  would 
be  better  served  by  taking  a  separate 
contracts  course.  The  ESH  and  Inter¬ 
national  Acquisition  lessons  were  short¬ 
ened  in  response  to  our  survey  results. 

Some  additional  material  on  operational 
cost  reduction  was  added  to  existing 
lessons,  but  this  area  requires  further 
work.  While  the  Ethics  lesson,  which  is 
based  on  the  Challenger  incident,  re¬ 
mains,  we  are  using  it  to  also  address  is¬ 
sues  of  effective  decision  making  in  ad¬ 
dition  to  purely  ethics  issues. 

The  Way  Ahead 

We  found  the  survey  on  systems  engi¬ 
neering  competencies  very  useful  in 
helping  us  develop  a  road  map  to  revise 
SYS-301.  While  many  of  the  results 
seem  intuitively  obvious,  it  was  impor¬ 
tant  to  have  inputs  from  a  broad  cross- 
section  of  the  acquisition  community 
before  w'e  proceeded.  We  did  find  some 
obvious  gaps  between  what  w'as  being 
taught  and  what  our  customers  felt  they 
needed.  Wfe  also  had  what  we  felt  was 
a  revelation  upon  seeing  the  significant 
gaps  between  the  perceived  importance 
of  a  set  of  competencies  and  the  per¬ 
ception  of  how  skilled  our  SPRDE  work¬ 
force  is  in  these  areas.  Clearly,  there  is  a 
great  need  for  continued  training. 


While  wre  changed  SYS-301  significantly, 
w*e  will  continue  to  make  additional 
changes  that  will  address  those  study  is¬ 
sues  that  have  not  yet  been  imple¬ 
mented.  Our  changes  will  also  include 
new  material  required  to  keep  the  course 
current  as  the  acquisition  environment 
continues  to  evolve. 

SYS-301  is  not  the  only  systems  engi¬ 
neering  course  to  have  undergone 
change.  In  June  of  2001  DAU  intro¬ 
duced  a  revised  Systems  201  (SYS-201) 
course — Intermediate  Systems  Planning, 
Research  Development  C. r  Engineering — 
that  converted  what  was  formerly  a  two- 
week,  in-residence  course  into  a  hybrid 
course  with  a  distance  learning  portion 
followed  by  one  week  in  class.  One  of 
the  options  we  are  considering  for  SYS- 
301  is  to  convert  it  to  a  hybrid  course 
in  the  future. 

Editor's  Note:  For  questions'comments, 
contact  Falk  at  iiiarlin.falk^dau.inil. 


FIGURE  3 
Gap  Analysis 

Total  Sys(«ems  View 
learning 
User  Fccus 

Cowart  Teclwcal 
Management 
Ctntiguraticn  Management 
Post  Prcdurton  Support 
Ein^iod  Maiagsmen! 
Operaiard  Cost  Reductnn 
Risk 

Management  ot  Chanang 
Tedirology 
Eaned  Value 
Software  Management 
Deygi  impacts  on 
Maiutartijrira 

ModelngardSlmJallon 

Ettics 

Erwrormenlaf  Safety  and 
Hedlh 

Eiterraticnai  Acqualnn 

Test  kite#  anon 
loestes  luegyation 
Eeduticnarv  Acqusifem 

Astimiatnn  ani 
CotTiruncalcn 
MeptabiUy 


104  PM  :  SEPTEMBER-OCTOBER  2002 


104 


Appendix  I 

LMI  WORKFORCE  REPORT  SPRDE/SE  FUNCTIONAL 

COMPETENCIES 


LMI  WORKFORCE  REPORT  SPRDE/SE  FUNCTIONAL  COMPETENCIES 


Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

RDT&E  Consolidation 
(Centers  of 

Excellence) 

Perform  Business  Case  Analysis 
(BCA)  (mission,  capabilities,  cost, 
trends,  future,  cross-Service 
opportunities  to  include  technical 
capabilities). 

Analyze  and  evaluate  different 
categories  of  data  such  as  cost  and 
technical  capabilities.  Analyze 
business  data  to  determine  its 
adequacy  and  impact  on 
consolidation  of  RDT&E  organization 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers 

Use  tradeoff  analyses  to  asses  most 
appropriate  consolidation 
recommendations 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Know  and  understand  service 
capabilities/core  competencies 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Know  critical  elements  contained  in  a 
Business  Case  Analysis  (BCA)  in 
order  to  justify  sound  business 
outcomes. 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Develop  streamlining  and 
implementation  planning  for 
consolidation 

Use  Business  Case  Analysis  (BCA)  to 
assess  effectiveness  of  the 
economies  of  budgeting  inherent 
government  functions  or  centers  of 
excellence  and  service  contracting  in 
the  business  sector. 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Know  and  understand  joint  and 
service  strategic  planning  and 
requirements  process;  assess  impact 
of  consolidation  options. 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  methods  for 
building  innovation  operations  that 
consistently  improve  overtime. 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Know  and  understand  the  Planning, 
Programming  and  Budgeting  System 
(PPBS)  and  DoD  monetary  policies 
and  procedures;  Assess  fiscal 
impacts  and  synthesize  consolidation 
options 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Know  effective  streamlining  and 
implementation  planning 
documentation 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Operate  in  a  Multi-Service 

Environment 

Know  and  understand  RDT&E 
process;  evaluate 
consolidation/process  change 
options;  synthesize  win-win  solutions 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Know  and  understand  virtual  RDT&E 
resources/network  applications;  ability 
to  assess  applicability  and  determine 
best  consolidation  applications 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Know  and  understand  how  to  function 
in  a  Multi-Service  environment. 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Understand  operation  systems  that 
effectively  connect  operations  with 
customers,  distribution  channels,  and 
suppliers 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers. 

Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Operate  in  Integrated  Product  and 
Process  Development  (IPPD) 
environment  in  developing  policy, 
provide  guidance  within  DoD/Industry 
groups  and  support  development  of 
performance  based  work  statements 
and  definition  of  performance  events. 

Critical  for  personnel  in  proposed 
consolidation  decisions.  Training 
should  be  specifically  focused  to  the 
Labs  and  Centers 

RDT&E  Increased 
Reliance  on  Non-DoD 
Organizations 

Conduct  market  research/analysis  of 
the  national  base  of  technology 

Understand  basic  market  research 
techniques 

Emphasis  needs  to  shift  to  enhance 
market  research  and  analysis 
techniques 

Know  technology  for  a  specific 
business  sector.  Understand  and 
evaluate  unique  conditions. 

Critical  for  those  in  labs  and  centers 

Assess  and  match  DoD/Non-DoD 
technical  and  facility  capabilities 
(retain  Smart  Buyer  Expertise  in  DoD- 
unique  areas. 

Know  and  understand  technology 
insertion  strategies  and  ability  to 
apply  to  DoD  needs. 

Critical 

Know  DoD’s  unique  technical  and 
facilitation  requirements 

Critical  skill  for  those  in  DoD  Labs  and 
Centers.  Current  training  needs  to  be 
expanded 

Assess  the  alternative  sources  and 
methods  most  appropriate  to  handle 
the  requirements  not  unique  to  DoD. 

Critical  skill  for  those  in  DoD  Labs  and 
Centers.  Current  training  needs  to  be 
expanded 

Know,  understand  and  be  able  to 
benchmark  and  evaluate  all  RDT&E 
options/  practices 

Critical  skill  for  those  in  DoD  Labs  and 
Centers.  Current  training  needs  to  be 
expanded 

Identify  appropriate  agreement 
method/vehicle  (CRDA,  MOA,  etc.)  to 
ensure  DoD’s  interests  are  protected 

Evaluate  the  individual  situation  and 
select  the  appropriate  contracting  or 
assistance  vehicle 

Have  knowledge  and  understanding 
of  appropriate  contracting  or 
assistance  vehicle. 

Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Understand  the  applicability  and 
advantages  of  the  various  contracting 
or  assistance  vehicles. 

Know  and  understand  dual  use 
application  of  program/item  needs 
and  ability  to  incentive  dual-use  focus 

Determine  the  most  appropriate 
agreement,  as  well  as  pricing  and 
facility  arrangements  that  will  mitigate 
risk  in  accordance  with  regulations, 
statutes  and  sound  business 
judgment 

Training  emphasis  needs  to  be  placed 
in  this  area.  Critical  for  S/E  in  Labs 
and  Centers 

Early  Involvement  of 
Operational  Test  and 
Evaluation 

Develop  Test  and  Evaluation  Master 
Plan  (TEMP)  to  allow  for  early 
involvement  of  T&E 

Determine  where  in  the  Test  & 
Evaluation  (T&E)  process  testing  can 
be  combined  to  ensure  greater 
participation  by  the  Operational 

Testers  up  front  while  maintaining 
their  independence. 

Understand  responsible  agencies  for 
Developmental  Test  &  Evaluation 
(DT&E),  Operational  Test  & 

Evaluation  (OT&E),  and  identify  the 
major  objectives  and  types  of 
development  and  operational  testing 

Training  emphasis  needs  to  be  placed 
in  this  area.  Critical  for  S/E  in  Labs 
and  Centers. 

Know,  understand  and  be  able  to 
operate  in  an  Integrated  Product 

Team  (IPT)  environment 

Training  emphasis  needs  to  be  placed 
in  this  area.  Critical  for  S/E  in  Labs 
and  Centers. 

Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Define  the  test  team  structure  and 
their  contributions  in  the  Test  & 
Evaluation  Master  Plan  (TEMP). 

Know  how  the  TEMP  is  used  as  an 
integrating  document  supporting  the 
acquisition  strategy  throughout  the 
entire  acquisition  life  cycle 

Training  emphasis  needs  to  be  placed 
in  this  area.  Critical  for  S/E  in  Labs 
and  Centers. 

Perform  design  tradeoffs  earlier  in 
the  acquisition  process. 

Know,  understand  and  be  able  to 
assess  design  tradeoffs 

Critical  to  engineering 

Evaluate  use  of  the  systems 
engineering  process  to  reduce  risk  of 
operational  /  support  problems 

Critical  to  engineering 

Develop  strategy  to  minimize 
operation/support  problems,  risks  and 
fielding  issues 

Understand  the  impact  of  design  on 
the  operations  and  test  environment 

Critical  to  engineering 

Plan  appropriate  T&E  of  commercial 
and  NDI  items. 

Know  propose  use  of  Commercial  & 
Non-Developmental  Items  (NDI)  be 
able  to  evaluate  such  items. 

Shift  training  emphasis  to  commercial 
/NDI  requirements.  Critical  to 
engineering. 

Apply  integrated  product  and  process 
development 

Understand  the  use  of  Integrated 
Product  and  Process  Development 
(IPPD)  in  successful  acquisition 
management 

Develop  verification  /  conformance 
metrics 

Be  capable  of  developing  strategic, 
tactical  and  local  metrics  with  in  the 
acquisition  process. 

Know  and  understand  metric 
development  and  linkage  to 
mission/operations  and  cost 
implications 

Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Integrate  verification/performance 
metrics  into  the  appropriate 
contacting  or  assistance  vehicle 

Knowledge  only. 

Know  and  understand  use  of 

Technical  Performance  Measures  and 
their  impact  on  cost  and  ability  to 
meet  contract  technical  requirements 

Increased  Use  of 
Simulation  Based 
Acquisition  (SBA) 

Perform  analysis  on  most  appropriate 
SBA  program  application,  select  pilot 
programs. 

Know  and  understand  potential 
DoD/Service  growth  areas  for 
application  of  Simulations  Based 
Acquisition  (SBA)  and  Modeling 
(specifically  O&S) 

Critical  for  engineering;  Training 
emphasis  needs  to  be  placed  in  M&S 
area. 

Understand  the  use  of  Modeling  and 
Simulation  (M&S)  across  the  total  life 
cycle  of  a  system 

Critical  for  engineering;  Training 
emphasis  needs  to  be  placed  in  M&S 
area. 

Use  SBA  to  identify  and  simulate 
design  issues  and  risks 

Ensure  risk  profile  are  analytically 
determined  using  proper  methods 

Critical  for  engineering;  Training 
emphasis  needs  to  be  placed  in  M&S 
area. 

Understand  the  factors  that  make  up 
the  simulation  model  and  verify  that 
logical  and  statistical  relationship 
exits. 

Critical  for  engineering;  Training 
emphasis  needs  to  be  placed  in  M&S 
area. 

Understand  and  determine  how  to 
apply  Modeling  and  Simulation  (M&S) 
when  conducting  performance 
studies,  effectiveness  studies, 
tradeoff  analysis,  risk  analysis, 
sensitivity  analysis  and  cost  analysis 

Critical  for  engineering;  Training 
emphasis  needs  to  be  placed  in  M&S 
area. 

Apply  simulation  and  modeling 
techniques 

Be  capable  of  using  and 
understanding  the  basic  tenets  of 
modeling  and  simulation 

Critical  for  engineering;  Training 
emphasis  needs  to  be  placed  in  M&S 
area. 

Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  the  types  of 
models  (physical,  mathematical, 
logical)  and  the  common  pitfalls  and 
limitations 

Critical  for  engineering;  Training 
emphasis  needs  to  be  placed  in  M&S 
area. 

Understand  the  methods  and 
simulations  associated  with  the 
processes  of  requirements 
generation,  program  management, 
design  and  engineering, 
manufacturing,  T&E,  logistics  support 
and  training 

Critical  for  engineering;  Training 
emphasis  needs  to  be  placed  in  M&S 
area. 

Separation  of  Tech 
Maturation  From 

Product  Development 

Perform  S&T  strategic  planning 

Know  and  understand  future 
technological  advances  that  can  be 
incorporated  into  system  development 
programs. 

Critical  for  SEs.  Incorporate  S&T 
planning  process  in  training. 

Know  and  understand  strategic 
planning  tools  and  techniques. 
Coordinates  with  Joint  War  fighting 
requirement. 

Critical  for  SEs.  Incorporate  S&T 
planning  process  in  training 

Conduct  risk  assessment,  risk 
reduction  and  mitigation  techniques  at 
the  earliest  possible  stage 

Know  and  understand  contingency 
planning  and  execution 

Assess  technological  opportunities 
and  evaluate  the  feasibility,  maturity, 
and  risk. 

Critical  for  SEs.  Training  emphasis  on 
current  tech. 

Know  and  understand  current  and 
future  science  and  technology 
research  and  development 

Critical  for  SEs.  Training  emphasis  on 
current  tech. 

Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  risk  reduction 
and  risk  assessment  processes  for 
acquisition  process.  Balancing  risks 
and  benefits  in  situations  involving 
human  safety,  environmental  risks, 
and  financial  uncertainties. 

Critical  for  SEs 

Develop  realistic  Technology 

Transition  Plans. 

Convert  Technology  Transition 

Planning  into  effective  and  executable 
contract  language 

Knowledge  only 

Know  and  Planning,  Programming 
and  Budgeting  System  (PPBS) 
environment  and  budgeting  process 
for  insertion  of  out  year  funds  for 
transition 

Knowledge  only 

Separation  of  Tech 
Maturation  From 

Product  Development 

Develop  realistic  Technology 

Transition  Plans 

Know  and  understand  technology 
transition  planning  /  strategy  and 
ability  to  assess  /  evaluate  and 
synthesize  best-value  options  into 
Technology  Transition  Plans 

Understand  the  risks  associated  with 
current  technology  maturity  in  relation 
program  needs 

Design  Systems  with  open 
architectures 

Know  and  understand  open 
architecture  discipline,  tools,  and 
methods  and  ability  to  apply  to 
service  interoperability 

Critical  for  SEs 

Conduct  affordability  assessments/ 
analysis. 

Be  able  to  do  parametric  analyses 

Shift  training  emphasis  to  performing 
Parametric  Analysis.  Critical  for 
engineering 

Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Understand  the  Cost  as  an 

Independent  Variable  (CAIV)  policy 
concerning  the  authority  of  the 
program  manager  to  make  cost  and 
performance  tradeoffs 

Knowledge  and  understanding  of 

CAIV  policy 

Know  and  understand  affordability 
assessment  techniques  and  tools 

Understand  theory  and  applications  of 
Integrated  Product  and  Process 
Development  (IPPD)  for  S&T 
programs  that  are  expected  to 
transition  to  the  next  phase  of 
acquisition.  Understand  how  to 
manage,  conduct  and  participate  in 
Integrated  Product  Teams  (IPTs). 

Assess  cost/schedule  risk  and 
influence  on  design 

Know,  understand  and  be  able  to 
perform  /  evaluate  cost/schedule  risk 
assessment 

Critical  skill  required 

Know  manufacturing  cost  implication 
resulting  from  product  designs 

Training  emphasis  on  this  area. 

Critical  skill 

Know  Cost  as  an  Independent 

Variable  (CAIV) 

Be  able  to  determine,  through  the  use 
of  risk  matrices,  templates,  or  maturity 
models,  the  influence  of  design  on 
risk 

Match  evolutionary  requirements  with 
mission  needs. 

Assess  technological  opportunities  an 
evaluate  the  feasibility,  maturity  and 
risk. 

1-10 


Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Understand  how  to  incorporate  a 
trade  study  methodology,  conduct  an 
analysis  and  provide  rationale  which 
address  tradeoffs  for  system 
requirements 

Critical  skill  required 

Assess  supportability  techniques  for 
assessing  systems  requirements 

Use  systems  engineering  processes 
to  reduce  risk  of  operational  and 
support  problems. 

Critical  skill  required 

Manage  experimentation  and 
prototyping 

Know  and  understand  supportability 
analysis  tools  and  techniques 

Training  emphasis  on  supportability 
analysis  tools.  Critical  skill  required 

Identify  the  impact  of  reliability, 
availability,  maintainability  on  system 
support  and  ownership  costs. 

Shift  training  emphasis  in  this  area  to 
address  TOC  impacts.  Critical  skill 
required. 

Identify  sources  and  methodologies 
for  technology  insertions 

Understand  commercial  and  military 
state  of  the  art  technology 
applications. 

Know  and  understand  open 
architecture  discipline,  tools,  methods 
to  improve  aging  systems/platforms 
O&S  (specifically  for  tech  insertions) 

Critical  for  SEs  at  labs  and  centers 

Know  methodologies  for  inserting 
technology  upgrades  and  maintaining 
technical  currency 

Apply  Advanced  Concepts  and 
Technological  Demonstrations 
(ACTDs)  as  appropriate  during 
product  development 

Know  and  understand  the  Advanced 
Concepts  and  Technological 
Demonstration  (ACTD)  process  and 
their  impact  on  Life  Cycle  Cost  (LCC) 

1-11 


Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Increased  emphasis 

On  Interoperability  As 

A  KPP 

Develop  systems  using  International 
Interoperability  Standards. 

Understand  the  increased  emphasis 
of  interoperability  as  a  Key 

Performance  Parameters  (KPP)  and 
ensure  it  is  reflected  in  the  solicitation 

Negotiate  in  the  international  political 
and  business  practices  environments 

Knowledge  only 

Identify  and  describe  basic  principles 
of  technical  standards  as  they  relate 
to  system  development  and 
interoperability 

Comply  with  Joint  Technical 
Architecture  requirements 

Knowledge  and  understanding  ability 
to  comply  with  Defense  Information 
Infrastructure  Common  Operating 
Environment  (Dll  COE) 

Knowledge  only 

Understand  and  apply  Joint  Technical 
Architecture  (JTA)  requirements  and 
standards 

Knowledge  only 

Perform  an  Interoperability 

Performance  Analysis. 

Perform  analysis  to  identify  linkages 
connections,  processes  and  delay 
time  that  effect  interoperability. 

Knowledge  as  opposed  to  perform 

Understand  framework  to  look  at 
interoperability  through  layers  such  as 
process,  software,  information  and 
influences 

Perform  a  Cost  as  an  Independent 
Variable  (CAIV)  analysis. 

Understand  the  purpose  and  general 
method  of  execution  of  Cost  as  an 
Independent  Variable  (CAIV) 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Prepare  and  defend  a  Cost  as  an 
Independent  Variable  (CAIV) 
analysis.  Discuss  the  relationship  of 
CAIV  analysis  to  other  cost  analysis 

Understand  the  Cost  as  an 

Independent  Variable  (CAIV)  policy 
concerning  the  authority  of  the 
program  manager  to  make  cost  and 
performance  tradeoff 

Increased  Emphasis 

On  Software 
Development 

Develop  evaluation  and  assessment 
criteria  to  measure  software  progress 

Know  and  understand  software 
evaluation  and  assessment  criteria 

Critical  for  engineers  working 
software  Knowledge  area  for  others. 

Understand  and  apply  the  software 
process  development  capability 

Critical  for  engineers  working 
software.  Knowledge  area  for  others. 

Know  and  understand 
customer/system  integration 
requirements  to  design  effective 
software  measures 

Critical  for  engineers  working 
software.  Knowledge  area  for  others. 

Understand  Capability  Maturity 

Models  (CMM) 

Critical  for  engineers  working 
software.  Knowledge  area  for  others 

Know  software  engineering  principles 
and  how  it  applies  through  the 
acquisition  life  cycle. 

Critical  for  engineers  working 
software.  Knowledge  area  for  others. 

Apply  parametric  analysis  for 
estimating  cost. 

Know  and  understand  parametric 
analysis  and  ability  to  perform  and 
analyze  resulting  data 

Critical  for  engineers  working 
software.  Knowledge  area  for  others. 

Understand  parametric  analyses  and 
construct  these  analyses  to  support 
bid  and  solicitation  development 

Critical  for  engineers  working 
software.  Knowledge  area  for  others. 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Apply  newly  developed  software 
evaluation  tools 

Know  evolutionary  spiral  process  as  a 
Framework  for  systems  and  software 
development  programs. 

Critical  for  engineers  working 
software  Knowledge  area  for  others. 

Understand  parametric  analyses  and 
construct  these  analyses  to  support 
bid  and  solicitation  development 

Critical  for  engineers  working 
software.  Knowledge  area  for  others. 

Know  evolutionary  spirals  process  as 
a  framework  for  systems  and  software 
development  programs. 

Critical  for  engineers  working 
software  Knowledge  area  for  others. 

Know  and  understand  software 
acquisition  risk  for  systems,  select 
appropriate  mitigation  strategies 

Critical  for  engineers  working 
software  Knowledge  area  for  others. 

Understand  the  software  development 
and  integration  process  and  the 
impacts  to  the  software  technical  life 
cycle 

Critical  for  engineers  working 
software.  Knowledge  area  for  others. 

Know  and  understand  the  integrated 
Capability  Maturity  Models  (CMM) 
process  and  how  it  applies  to 
software  development 

Critical  for  engineers  working 
software,  Knowledge  area  for  others. 

Know  and  understand  leading/  state 
of  art  software  evaluation  “best 
practices”  and  resources  for  software 
test  program  planning  and  execution 

Shift  training  to  state  of  art  evaluation 
best  practices.  Critical  for  S/W 
engineering 

Be  able  to  use  and  illustrate  state  of 
art  tools  and  techniques  available  for 
planning,  measuring  and  predicting 
software  development  progress 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

O&S  Consolidation 

Take  joint  or  corporate  approaches  to 
DoD  sustainment  issues  (corporate 
contracts,  standard  corporate 
information  systems.) 

Know  and  understand  environmental 
rules/regulations 

Knowledge  only 

Know  and  understand  environment 
assessment  to  law,  policy, 
regulations,  community  impact  and 
issues/impediments  to 

Knowledge  only 

Know  and  understand  commercial 
best  practices 

Knowledge  only 

Perform  /  Assess  business  case 
analysis  (mission,  capabilities,  costs, 
trends,  opportunities.) 

Know  and  understand  risk 
assessment  methods  and 
measurement  tools 

Know  and  understand  Business  Case 
Analysis  (BCA)  process/rules  and 
tools 

Develop  streamlining  and 
implementation  planning  for 
consolidation 

Know  and  understand  organizational 
processes  and  measurement 

Knowledge  only 

Know  and  understand  personnel 
policies  /  procedures  to  include  labor 
relations/union  coordination 

Knowledge  only 

Ensure  highest  quality  staff 
infrastructure  is  maintained 

Know  and  understand  available 
training/education  resources  to 
include  funding/  opportunity 

Reengineer  the 

Product  Support 

Process  to  Use  Best 
Practices 

Benchmark  government  and  industry 
to  identify,  adopt,  and  tailor  best 
practices 

Know  and  understand  commercial 
best  practices 

Knowledge  only 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  analyzing 
lessons  learned,  particularly  from 
recent  reengineering  efforts. 

Know  and  understand  benchmarking 

Knowledge  only 

Know  and  understand  research 
methods  literature,  internet,  corporate 
assets 

Knowledge  only 

Perform  business  case  analysis 

Know  and  understand  Business  Case 
Analysis  (BCA)  process/rules  and 
tools 

Knowledge  only 

Involve  customers  early  in  the 
acquisition  strategy  process 

Be  able  to  identify  and  correct 
potential  imbalances  in  level  playing 
field  involved  in  public-private 
competitions 

Know  and  understand  acquisition 
process 

Know  and  understand  the  Planning, 
Programming  and  Budgeting  System 
(PPBS) 

Understand  operating  systems  that 
effectively  connect  operations  with 
customers,  distribution  channels,  and 
suppliers 

Knowledge  only 

Employ/Develop  sourcing  strategies 
that  emphasize  best  value 

Know  and  understand  customer 
requirements 

Ability  to  develop  performance-based 
work  statements  or  statements  of 
objectives 

Critical  for  those  developing  work 
statements  otherwise,  knowledge  only 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  risk 
assessment  techniques  measurement 
tools,  cost  benefit  analysis,  and  fault 
tree  methods  for  describing  and 
making  decisions 

Know  and  understand  current  product 
support  processes  and  interrelations 

Knowledge  only 

Know  and  understand  commercial 
best  practices 

Knowledge  only 

Know  and  understand  data  analysis 
to  include  cost//performance  tradeoffs 
in  order  to  achieve  affordable 
readiness 

Develop  performance-based  work 
statements  or  statements  of 
objectives 

Know  and  understand  product 
/service  to  be  supported 

Knowledge  only 

Know  and  understand  performance- 
based  work  statements  or  statement 
of  objectives  development  / 
environment 

Critical  for  those  developing  work 
statements  otherwise  knowledge  of 

Develop  incentives  for  public  and 
private  sources  to  provide 
sustainment  support  in  a  timely  and 
efficient  manner  while  reducing  TOC 

Know  and  understand  process 
change  enablers 

Know  and  understand  components  of 
Total  Ownership  Cost  (TOC) 

Critical  skill  required 

Apply  integrated  supply  chain 
practices 

Seek  integrated  Supply  Chain 
Management  (SCM)  solutions 

Knowledge  only 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  ways  to  apply 
Single  Process  Initiative  (SPI)  to 
optimize  logistics  operations 

Knowledge  only 

Know  and  understand  Supply  Chain 
Management  (SCM)  purposes  and 
processes  -  components  and  total 

Knowledge  only 

Apply  Supply  Chain  Management 
(SCM)  processes/  methods  to 
business  opportunity/  situation 

Knowledge  only 

Expansion  of  Prime 
Vendor/  Virtual  Prime 
Vendor /PV-VPV  like 
arrangements 

Analyze  markets  not  currently 
covered  by  prime  vendor-type 
arrangement,  and  develop  acquisition 
strategies 

Know  and  understand  commercial 
practices,  including  best  practices  of 
specific  market  sector 

Knowledge  only 

Tailor  the  application  of  best  practices 
to  the  specific  market  sector  and 
envelop  appropriate  contractual 
vehicles 

Know  and  understand  commercial 
best  practices 

Knowledge  only 

Increased  Contractor 
Logistics  Support 

Develop  integrated  support  strategies 

Know  and  understand  common 
support  requirements  and  tools  and 
ability  to  leverage  those 
opportunities. . .consolidated  design 
and  buying  opportunities 

Know  and  understand 
sustainment/war  reserve 
requirements 

Knowledge  only 

Know  and  understand  alternative 
logistics  support  risks,  costs,  and 
performance  in  development  of 
integrated  support  strategies  and 
tools. 

Knowledge  only 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  sustainment 
processes/techniques  and  process 
interrelationships  (specific  emphasis 
on  inventory  management  with  a 
configuration  management  subset 

Knowledge  only 

Develop  and  apply  rules  and  tools  to 
help  determine  the  best  application  of 
contractor  support  and  best  use  of 
innovative  contracting  techniques  for 
common  support  requirements. 

Know  how  to  develop  and  apply  rules 
and  tools  to  help  determine  the  best 
application  of  contractor  support  and 
best  use  of  innovative  contracting 
techniques  for  common  support 
requirements. 

Knowledge  only 

Outsource  Equipment 
Disposal  Activities 

Conduct  capability/environmental 
assessment. 

Know  how  to  identify  hazardous 
property  and  recognize  the  existence 
of  federal,  state  and  local 
requirements  that  may  impact  on  its 
disposal  in  accordance  with  NEPA, 
RCRA,  TSDA,  FAR  AND  DFARS. 

Additional  training  emphasis  on  this 
area. 

Know  and  understand  environmental 
regulations  and  cost  assessments. 

Knowledge  only 

Assess  contractor’s  security 
processes  and  procedures. 

Know  demilitarization  requirements  to 
assure  resale  of  surplus  material 
eliminates  potential  of 
hazardous/safety  incidents. 

Critical  for  engineering  involved  in 
safety.  Training  emphasis  needs  to 
occur  in  this  area 

Increased  Use  of 

Vendor  Managed 
Inventory,  Direct 

Vendor  Delivery,  and 
Time-Definite  Delivery 

Use  mature,  robust,  market-ready 
commercial  capabilities  with  end-to- 
end  visibility  of  inventory. 

Understand  and  apply  past 
performance  in  structuring  of  a 
solicitation. 

Knowledge  only 

Know  and  understand  current, 
market-ready  commercial  practices 
with  end-to-end  visibility  of  inventory. 

Knowledge  only 

1-19 


Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Increased  PM 

Influence  to  Reduce 
TOC  (with  specific 
emphasis  on  funding 
issues) 

Emphasize  life-cycle  cost  implications 
in  all  program  management  phases 
and  decisions. 

Know  and  understand  life-cycle 
management  processes  and  phases. 

Understand  Cost  as  an  Independent 
Variable  (CAIV). 

Know  and  understand  contracting 
options  available. 

Know  and  understand  formal  and 
informal  organizational  structure  to 
generate  best  value  solutions  to 
reduce  Total  Ownership  Cost  (TOC) 

Know  and  understand  weapon 
system/platform  mission/operating 
environment. 

Analyze  market  research/customer 
requirements/sourcing  strategies  to 
synthesize  best  value  solutions. 

Know  and  understand  risk 
assessment  techniques  and 
measurement  tools. 

Know  and  understand  commercial 

Total  Ownership  Cost  (TOC)  and  life- 
cycle  practices  and  tools. 

Know  and  understand  the  Planning, 
Programming  and  Budgeting  System 
(PPBS)  and  defense  fiscal 
management  policies  and  practices. 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  Total 

Ownership  Cost  (TOC)  variables  and 
impacted  areas,  to  include  how  to 
perform  tradeoff  analyses  of  capability 
performance  and  life-cycle  cost 
drivers  and  evaluation. 

Develop  or  modify  oversight 
processes  and  analysis  tools. 

Know  cost  estimating  methods. 

Know  which  funding  accounts  the 
Program  Manager  must  influence  to 
reduce  Total  Ownership  Cost  (TOC). 

Know  and  understand  commercial 
industry  analysis  tools  and  oversight 
processes. 

Know  cost  models,  contractor 
systems  and  process  risks. 

Understand  operating  and  support 
cost  data  and  data  sources  (e.g., 
Service  VAMOSC  Systems)  and  their 
differences;  cost  estimating 
tools/models  and  their  limitations. 

Know  and  understand  existing 
oversight  processes  and  analysis 
tools. 

Understand  Total  Ownership  Cost 
(TOC)  from  several  O&S  perspectives 
(e.g.,  weapon  systems,  units  and 
organizations). 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Perform  trade-off  analysis  of 
capability,  performance,  and  life-cycle 
cost  considerations. 

Know  and  understand  life-cycle 
management  processes  and  phases. 

Knowledge  only 

Know  and  understand  contingency 
planning  through  plan  development 
and  implementation. 

Knowledge  only 

Know  and  understand  analysis 
techniques  and  tools. 

Knowledge  only 

Analyze  market  research/customer 
requirements/sourcing  strategies  to 
synthesize  best  value  solutions. 

Knowledge  only 

Know  and  understand  contract 
incentives/disincentives  through 
contract  completion. 

Knowledge  only 

Use  of  Electronic 
Commerce  and  Other 
Information 

Technology 

Use  web-based  acquisition  systems 
(e.g.,  electronic  catalogs,  DoD  E-Mail) 

Know  and  understand  electronic 
business  environment  (e.g.,  Internet, 
Word  Wide  Web  and  Intranet  Tools 
and  Applications) 

Knowledge  only 

Require  business  partners  to  apply 
electronic  commerce  techniques  and 
tools. 

Understand  DoD  electronic 
commerce  policy. 

Know  and  understand 
marketing/selling  methods  and 
strategies. 

Increase  Competitive 
Sourcing  of  Services 

Determine  acquisition  strategy  (e.g., 
regional,  omnibus). 

Know  and  understand  strategic 
planning.  Know  how  to  develop 
acquisition  strategy. 

Conduct  Best  Value  Analysis  on 
services/cost. 

Know  and  understand  Best  Value 
Analysis  and  understand  how  to  apply 
in  source  selections. 

Knowledge  only.  Critical  for  those 
involved  in  source  selections. 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Integrated  Digital 
Environment 

Leverage  commercial  technology  to 
support  modern  business  operation 
(e.g.,  virtual  office 

Know  and  understand  hardware, 
software,  and  network  requirements 
and  applications  and  interoperability. 

Knowledge  only 

Know  and  understand  Internet,  World 
Wide  Web  and  Intranet  Tools  and 
Applications. 

Knowledge  only 

Know  and  understand  electronic 
commerce  system  relationship  to 
existing  business  process  and  their 
interrelationships. 

Knowledge  only 

Know,  understand  and  be  able  to 
apply  business  process 
reengineering. 

Knowledge  only 

Know  and  understand 
statutory/regulatory  environment 

Knowledge  only 

Know  and  understand 
marketing/selling  methods  and 
strategies. 

Knowledge  only 

Know  and  understand  performance 
metrics. 

Knowledge  only 

Know  and  understand  enterprise 
resource  planning  concepts  and 
solutions. 

Knowledge  only 

Know  and  understand  commercial 
electronic  commerce  processes 

Knowledge  only 

Develop  affordable  requirements 
document  for  establishing 
software/hardware  architecture  for 
Integrated  Digital  Environment  (IDE) 

Critical  for  those  generating 
requirements. 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Use  commercial  standard  reference 
identification  number  system  to 
simplify  with  private  and  federal 
sectors  (e.g.,  Central  Contractor 
registration). 

Know  and  understand  unique 
software  requirements  and 
applications. 

Critical  for  those  working  S/W 
requirements.  Knowledge  for  all 
others. 

Know,  understand  and  be  able  to 
incorporate  standards  into  paperless 
operating  systems  of  commercial 
standard  reference  identification 
number  system. 

Knowledge  only 

Apply  existing  national  and 
international  standards,  practices  and 
technologies  to  automate  the 
management  and  exchange  of 
information 

Know  and  understand  national  and 
international  standards,  practices  and 
technologies  used  to  automate  and 
manage  and  exchange  information. 

Knowledge  only.  Critical  for  those 
working  standards  program. 

Achieve  Paperless 
Contracting 

Use  electronic  mediums  for  electronic 
payments. 

Know  and  understand  electronic 
mediums  for  electronic  payment. 

Knowledge  only 

Know  and  understand  strengths  and 
weaknesses  of  integration 

Knowledge  only 

Recognize  Government  and 
commercial  cultures  to  effectively 
educate/market/encourage 
commercial  participation 

Knowledge  only 

Use  purchase  cards,  electronic 
catalogs,  electronic  commerce  and 
imaging. 

Recognize  statutes,  rules,  policies, 
and  procedures. 

Knowledge  only 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Introduction  and 
Maturation  of 

Knowledge 

Management 
Techniques  and 
Practices 

Improve  data  management  and 
availability  (within  government  and 
between  government  and  industry). 

Know,  understand  and  be  able  to 
evaluate  and  apply  knowledge  and 
data  management  tools  and 
techniques  to  paperless  acquisition. 

Knowledge  only 

Security/Proprietary 

Information 

Use  adequate  security  measure  (to 
include  protocols)  to  protect  electronic 
information,  as  appropriate,  and  to 
provide  ready  access. 

Know  and  understand  security 
statutory/regulatory  environment 

Knowledge  only 

Know  and  understand  adequate 
security  measures. 

Knowledge  only 

Increased  Commercial 
Military  Integration 

Promote  use  of  commercial  items 

Perform  advanced  market  research  of 
commercial  and  military  products 

As  it  applies  to  SPRDE,  only  the 
performance  of  basic  market  research 
should  be  expected. 

Know,  understand  and  be  able  to 
understand  benefits  of  opportunities 
of  using/transitioning  to  commercial 
items  where  available. 

Knowledge  only 

Know  and  understand  commercial 
and  MILSPEC  systems  and  specific 
sector  practices;  ability  to  develop 
solutions  and  deconflict  to  eliminate 
military  specifications  where 
commercial  products  and  practices 
fulfill  the  requirement. 

Knowledge  only 

Know  Part  12  and  the  direct  impact 
on  insight  and  business/technical 
processes  relationship  with  defense 
contractors. 

Critical  skill 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  operational  organizations, 
concepts  and  data  sources. 

Knowledge  only 

Perform  an  analysis  of  alternatives. 

Critical  Skill 

Analyze/challenge  requirement  in 
order  to  accept  commercial  items. 

Critical  Skill 

Develop  and  maintain  knowledge  of 
the  commercial/industrial/academic 
sectors. 

Critical  for  S&E 

Participate  in  sector  activities  (e.g., 
professional  associations) 

Know  and  understand  sector 
resources,  activities,  world-class 
practices,  process,  technologies,  and 
integration  impediments. 

Be  able  to  synthesize  Government 
and  commercial  cultures  to  effectively 
educate/market/encourage 
commercial  participation  in  CMI. 

Knowledge  only 

Increased  Use  of 
Common  Business 
Practices 

Promote  use  of  common  business 
practices 

Know  and  understand  benchmarking 
methods  and  tools. 

Knowledge  only 

Know  and  understand  industry  data 
exchange  programs. 

This  is  GIDEP 

Identify  and  adapt  common  and/or 
better  practices. 

Know  and  understand 
warranties/guarantees  and  ability  to 
synthesize  into  Government  system. 

Apply  and/or  tailor  commercial 
business  sector  practices  (e.g.,  Single 
Process  Initiative  (SPI)). 

Knowledge  only 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Perform  advanced  market  research  of 
commercial  and  military  products. 

Employ  Common 
Technology  Bases 

Promote  knowledge  of  world-class 
technology  bases 

Know  and  understand  potential 
DoD/Service  growth  areas  of 
application  of  CM  I/technology  bases 
(specifically  O&S). 

Critical  for  SEs  directly  involved. 
Knowledge  only  for  others 

Develop  knowledge  of  relevant 
technology  bases,  resources,  and 
capabilities. 

Critical  for  SEs  directly  involved. 
Knowledge  only  for  others 

Participate  in  technology  sector 
activities. 

Know  and  understand  sector 
resources,  activities,  world-class 
practices,  processes,  technologies, 
and  integration  impediments. 

Critical  for  SEs 

Know  and  understand  dualOuse 
application  of  program/item  needs 
and  ability  to  incentives  dual-use 
focus. 

Critical  for  SEs  involved  in  tech 
transition  and  dual  use  programs. 

Employ  Flexible 
manufacturing 
(Economic 
manufacture  of 

Varying  Size  and 

Types) 

Employ  flexible  manufacturing 

Know  and  understand  flexible 
manufacturing  techniques  and  tools 
(e.g.,  CAD/CAM) 

Evaluate  adequacy  of  workload 
planning. 

Knowledge  only 

Evaluate  adequacy  of  contractor 
manufacturing  capabilities 

Knowledge  only 

Know  and  understand  agile 
manufacturing 

Knowledge  only 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  surge 
manufacturing  and  ability  to  develop 
best  solution  for  CMI  when  factoring 
in  surge  requirements 

Knowledge  only 

Know  and  understand  Diminishing 
Manufacturing  Sources  (DMS) 
commodities 

Know  and  understand  and  ability  to 
evaluate  adequacy  of 
government/contractor  manufacturing 
capabilities/flexible  manufacturing 
tools. 

Knowledge  only 

Coordinates  supply  chain  requirement 
(consider  and  integrate  all  phases  in 
manufacturing  flow). 

Know  and  understand  Supply  Chain 
Management  (SCM)  practices  and 
tools. 

Knowledge  only 

Be  able  to  develop  and  evaluate 

Supply  Chain  Management  (SCM) 
options  for  CMI 

Knowledge  only 

Extend  MILSPEC/ 
MILSTANDARD 

Reform  to 
Re-procurements 

Reduce  MILSPEC/MILSTANDARDS 
in  re-procurements 

Develop  performance-based 
specifications. 

Critical  for  SEs 

Know,  understand  and  be  able  to 
determine  if  CMI  or  military  spec  is 
applicable. .  .safety/health/mission 
needs. 

Critical  for  SEs 

Develop  sources  as  required. 

Critical  for  SEs 

Know  and  understand  quality  and 
testing  needs/requirements 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  customer 
requirements 

Perform  market  analyses. 

Manage  multiple  configurations. 

Know  and  understand  commercial 
and  MILSPEC  systems  and  specific 
sector  practices;  ability  to  develop 
solutions  and  deconflict  to  eliminate 
military  practices  fulfill  the 
requirement 

Life  Cycle/Reduced 

Total  Ownership  Cost 
Emphasis 

Reduce  Life  Cycle  Cost/Total 
Ownership  Cost 

Apply  Cost  as  an  Independent 

Variable  (CAIV)  and  reduced  Total 
Ownership  Cost  (TOC). 

Identify,  analyze  and  manage  Life 

Cycle  Cost  (LCC)  drivers. 

Know  and  understand  cost  analysis 
and  life-cycle  management. 

Establish  activity  based  costing  for 
the  life  cycle  process. 

Comprehend  DoD’s  corporate 
implementation  f  activity  based 
costing  and  management 

Evolutionary 
Acquisition/Reduced 
Cycle  Time 

Promote  evolutionary  and  incremental 
acquisition  as  appropriate 

Perform  risk  analysis. 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know,  understand  and  be  able  to 
assess,  evaluate  and  synthesize 
evolutionary/incremental  enablers 
(e.g.,  open  architecture, 
interoperability,  CMI,  ACTD, 
technology  insertion  into 
comprehensive,  cost-effective 
solution. 

Assess  and  forecast  technology 
maturation  for  system  insertion. 

Analyze  and  evaluate  requirements 
for  validity  of  evolutionary  and 
incremental  acquisitions. 

Minimize  cycle  time 

Evaluate  technology  maturation  to 
support  short  cycle  time  in  produce 
development. 

Know  and  understand 
benchmarking/metrics  analysis  and 
ability  to  apply  and  evaluate  in 
acquisition  process  to  baseline  and 
reduce  cycle  time. 

Understand  technology  maturation  vs. 
product  application. 

Flexible  User 
Requirements 

Participate  in  development  of  user 
requirements. 

Perform  risk  analysis. 

Know  operational  organizations, 
concepts  and  date  sources. 

Perform  Risk  Based  Surveillance 

Critical  Skill 

Know  and  understand  user  and  joint 
operating  requirements. 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Define  and  analyze  alternatives. 

Know,  understand  and  be  able  to 
operate  in  Integrated  Product  Team 
(IPT)  environment. 

Technology 

Refreshment  of 

System  (Modernization 
through  Spares) 

Promote  technology  refreshment  of 
systems. 

Know  and  understand  inventory 
management  methods  and  practices 
and  interrelationships  to  inventory 
re-procurements. 

Knowledge  only 

Know,  understand  and  be  able  to 
assess,  evaluate,  and  synthesize 
technology  refreshment  enablers 
(e.g.,  open  architecture, 
interoperability,  CMI,  ACTD, 
technology  insertion  into 
comprehensive,  cost-effective 
solution) 

Assess  and  forecast  technology 
maturation. 

Apply  open  systems  architectures. 

Know  and  understand 
interchangeability/interoperability  and 
substitution. 

Identify,  analyze  and  manage  Life 

Cycle  Cost  (LCC)  drivers. 

Develop  performance  based 
specifications 

Know,  understand  and  be  able  to  use 
engineering  change  process  methods 
and  tools. 

Know,  understand  and  be  able  to  use 
value  engineering  methods  and  tools. 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Know  and  understand  motivation 
techniques  to  incentivize  to  develop 
product  improvements 

Evaluate  performance-based  work 
statements  and  advise  program  office 
as  appropriate. 

Obtain  and  execute  funding  for 
modernization. 

Synthesize  the  functions  of  the 
Planning,  Programming  and 
budgeting  System  (PPBS). 

Knowledge  only 

Increased  Scope  of 
Other  Transactions 

Expand  use  and  scope  of  other 
transactions. 

Manage/overseas  other  transactions. 

Know  and  understand  potential 
DoD/Service  growth  areas  for 
application  of  other  transactions 
(specifically  O&S). 

Define,  select  and  adapt  terms  to  the 
specific  agreement. 

Perform  advanced  market  research. 

Know,  understand  and  be  able  to 
access/evaluate  and  synthesize  data 
of  Other  Transaction  Authority  (OTA) 
low/policy/resources. 

Increased  use  of  Best 

value-Dissimilar 

Competition 

Expand  use  of  best  value  and 
dissimilar  competitions  including 
capabilities  tradeoff  vs.  mission. 

Develop  performance  based 
solicitation. 

Apply  modeling  and  simulation 
techniques. 

Analyze  expected  system 
performance  outcomes  for  best  value. 
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Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Analyze  user  requirements. 

Increased  Use  of 
Performance  Based 
Contracting 

Capitalize  on  opportunities  to  develop 
performance  based  solicitations  for 
products  and  services. 

Develop  performance  expectations 
incentives  and  metrics  to  describe 
acquisition  needs  and  evaluate 
outcomes. 

Know  and  understand  sector 
resources  and  activities. 

Know  and  understand  common 
business  practices. 

Know  and  understand  world-class 
sector  practices  processes  and 
technologies  (e.g.,  Single  Process 
Initiative  (SPI)). 

Increased 

Collaboration  Between 
User  and  Acquisition 
Communities 

Promote  collaboration  between  user 
and  acquisition  communities 

Know  and  understand  the  Integrated 
Product  Team  (IPT)  environment. 

Be  able  to  synthesize  Government 
and  commercial  cultures  to  effectively 
educate/market/encourage 
collaboration  between  user  and 
acquisition  communities. 

Knowledge  only 

Promote  collaboration  between  user 
and  acquisition  communities. 

Know  and  understand  collaboration 
impediments. 

Knowledge  only 

Develop  mutual  understanding  of  user 
roles  and  functions  and  the 
acquisition  system  capabilities. 

Knowledge  only 

Model  and  analyze  manufacturing 
system  performance. 

Knowledge  only 

Environmental  Trend 

Functions 

ID  Competency 

Comments/Nuances 

Maintain  open  communications 
through  all  phases  of  the  life  cycle. 

Appendix  J 

Future  Acquisition  and  Technology  Workforce  Report  (April  2000)  - 

Updated  Assumptions 


LMI  ASSUMPTIONS 


Future  Acquisition  and  Technology 
Workforce  Report  (April  2000) — 
Updated  Assumptions 


Institute  for  Defense  Analyses 
3  March  2004 


Basis  for  Your  Review  of  the  Following 
Charts 

•  IDA  is  conducting  a  study  to  identify  relevant 
SPRDE/SE  duties  and  tasks  (competencies) 
with  which  to  conduct  a  review  of  the  SPRDE/SE 
courses 

•  The  Future  Acquisition  and  Technology 
Workforce  Report,  published  in  early  2000, 
contains  certain  assumptions  (called  trends) 
upon  which  its  lists  of  competencies  are  based 

•  IDA  wants  to  validate,  change,  or  eliminate 
these  various  assumptions  based  on  the  current 
environment  to  determine  how  to  include  the 
identified  competencies  in  their  study 

•  We  would  like  your  input 
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Overview  of  the  Workforce  Report 

•  Future  trends  are  identified,  both  global  and 
functional,  that  will  effect  what  the  future 
workforce  will  be  required  to  do  (functions)  and 
what  skills  it  will  need  (competencies) 

-  -100  future  functions  are  identified — activities  the 
workforce  must  perform  to  implement  acquisition 
reforms  and  new  practices 

-  27  universal  competencies  and  traits  are  identified  as 
being  needed  by  the  entire  workforce,  to  differing 
degrees  depending  on  job  level 

-  Over  400  detailed  functional  competencies  were 
developed,  sorted  by  career  field 
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First,  Environmental  Trends  drove 
the  Master  Functional  Competencies 

Only  those  trends  that  influenced 
the  SPRDE  competencies  are 
included  in  the  next  5  slides 
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Research  Development,  Test  & 
Evaluation 

•  Consolidation  (Centers  of  Excellence) 

•  Increased  reliance  on  non-DoD  organizations 

•  Early  involvement  of  OT&E 

•  Increased  use  of  M&S 

•  Separation  of  tech  maturation  from  product  development 

•  Increased  emphasis  on  interoperability  as  a  KPP 

•  Increased  emphasis  on  software  development 

•  Increased  reliance  on  performance-based  specs 

•  Increased  emphasis  on  war  fighting  “capabilities”  vice 
detailed  system  requirements 
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Operations  &  Support 

•  Consolidation 

•  Reengineer  the  product  support  process  to  use  best  practices 

•  Expansion  of  prime  vendor/virtual  prime  vendor-like  arrangements 

•  Increased  contractor  logistics  support 

•  Outsource  equipment  disposal  activities 

•  Increased  use  of  vendor-managed  inventory,  direct  vendor  delivery,  and  time- 
definite  delivery 

•  Increased  PM  influence  to  reduce  TOC  (with  specific  emphasis  on  funding 
issue) 

•  Use  of  electronic  commerce  and  other  information  technology 

•  Increase  competitive  sourcing  of  services 

•  Increasing  influence  of  considerations  such  as  technology  refreshment  for 
COTS 

•  Faster  changes  in  the  system  physical  baselines  -  re-engineering  of  the 
configuration  management  paradigm 

•  Emphasis  on  product  quality 


9/8/2004  6 


J-3 


Move  to  Paperless  Acquisition 

•  Integrated  digital  environment 

•  Achieve  paperless  contracting 

•  Introduction  and  maturation  of  knowledge 
management  techniques  and  practices 

•  Security/proprietary  information 
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Emphasize  Commercial-Military 
Integration  (CMI) 

•  Increased  CMI 

•  Increased  use  of  common  business 
practices 

•  Employ  common  technology  bases 

•  Employ  flexible  manufacturing  (economic 
manufacture  of  varying  sizes  and  types) 

•  Extend  MILSPEC/MILSTANDARD  reform 
to  re-procurements 

9/8/2004  8 
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Adopt  New  Approach  to  Acquisition 

•  Life  cycle/reduced  total  ownership  cost 
emphasis 

•  Evolutionary  acquisition/reduced  cycle  time 

•  Flexible  user  requirements--focus  on  system 
and  warfighter  capabilities 

•  Technology  refreshment  of  systems 
(modernization  through  spares) 

•  Increased  use  of  other  transactions 

•  Increase  use  of  best  value-dissimilar  competition 

•  Increased  use  of  performance-based  contracting 

•  Increased  collaboration  between  user  and 
acquisition  communities 
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Second,  additional  SPRDE- 
specific  competencies  were 
developed 

Based  on  the  following  vital 
competencies  and 
global/functional  trends 
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SPRDE  Vital  Competencies 

•  Ability  to  operate  in  an  IPPD  environment 

•  Understanding  and  mastering  competencies  to  control  TOC 

•  Ability  to  understand  and  use  the  SE  process  to  develop  balanced  system 
solutions  that  meet  requirements,  reduce  risk, -control  cost, -and  exhibit 
attributes  of  robustness  throughout  the  acquisition  life-cycle 

•  Ability  to  perform  parametric  analyses  for  both  hardware  and  software 
systems 

•  Understand  what  M&S  tools  are  available  and  how  to  best  use  them 

•  Understanding  and  proper  use  of  open  systems  and  open  architectures 

•  Evaluation,  assessment,  development,  and  integration  of  software 
products/efforts  that  support  the  acquisition  of  new  systems 

•  Understand  how  to  perform  comprehensive  risk  analyses  and  risk 
assessments 

•  Ability  to  address  Information  Assurance  requirements  throughout  the 
acquisition  life-cycle 

•  Ability  to  estimate  cost 
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Impact  of  Global/Functional  Trends 
on  SPRDE  Career  Field 

•  Smaller  workforce 

•  Older  workforce 

•  Information  technology 

•  Knowledge  management  and  learning  organization 

•  Cross-functional  teaming  and  personnel  mobility 

•  Competitive  sourcing 

•  Ever-increasing  product  complexity 

•  Pressure  for  shorter  development  time 

•  Need  to  control  costs 

•  Over-optimism 
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Finally,  a  vision  for  SPRDE  was 
developed 

The  vision  is  for  2005,  and  we  are 
now  in  2004 


SPRDE  Career  Field  Vision 


In  2005  based  on  the  global  and  functional  trends  ,  government  engineers  will  either  be 
part  of  IPTs  or  themselves  will  form  small  IPTs  that  cover  critical  engineering  competency 
areas.  The  team  will  be  able  to  operate  in  a  performance-based  environment  and  will 
have  the  skills  necessary  to  help  customers  meet  cost,  schedule  and  technical  targets. 
They  will  have  a  thorough  understanding  of,  and  maintain  competency  in 

-  Systems  and  design  engineering 

-  Risk  management 

-  Logistics 

-  Test 

-  Manufacturing 

-  Quality  Assurance 

-  Modeling  and  simulation 

-  Open  systems 

-  Software 

And  their  interactions,  integration,  and  effect  on  cost,  schedule,  and  technical 
performance.  Engineers  will  be  well  versed  in  commercial,  industrial,  and  academic 
technology  and  management  developments.  Small  teams  of  engineers  will  be  able  to 
react  quickly  to  customer  needs,  perform  the  necessary  analyses,  and  make  proper 
recommendations  to  help  preclude  or  fix  technical  proqram,  system,  or  process 
problems. 
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Please  direct  your  comments  as  to 
whether  these  assumptions  are  still 
valid  and/or  whether  any 
changes/additions  should  be  made. 

krichter@ida.org 

THANKS 
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Appendix  K 

SPRDE/SE  Duties  and  Tasks  for  SYS  101,  201  and  301 


SPRDE/SE  DUTIES  AND  TASKS— SYS  101 


These  duties  and  tasks  are  divided  by  shaded  headings.  The  major  headings  are 
highlighted  in  dark  gray:  SE  Processes,  Design  Considerations,  and  SE  Techniques  and 
Tools.  The  sub-headings  are  shown  in  lighter  gray. 

SYS  101-The  Systems  Engineering  Processes 

Define  systems  engineering. 

Know  where  to  go  for  SE  guidance,  e.g.,  PM  CoP,  INCOSE,  guides  and  handbooks. 

SE  Processes 

Apply  SE  processes  at  the  initial  stages  of  program  formulation  and  throughout  all  phases  of  the  system 
lifecycle. 

Apply  systems  engineering  throughout  the  total  system  lifecycle. 

Understand  the  requirements  for  early  integration  of  ESOH  considerations  into  SE. 

Recognize  that  SE  is  the  process  of  technical  management  in  the  defense  environment,  and  how  it  is 
used  in  translating  operational  needs  and  requirements  into  an  integrated  system  design  solution. 

Relate  how  SE  processes  are  applied  within  Integrated  Product/Process  Development  (IPPD)  to  translate 
customer  needs  into  system  products  and  processes. 

Translate  the  JCIDS-defined  capabilities  to  a  system-specific  design  solution  and  understand  the  major 
goals  of  this  process. 

Apply  the  SE  processes  to  transform  requirements  and  constraints  into  an  operational  system  design. 

Understand  the  national  and  international  SE  process  standards  and  models  and  how  to  use  them. 

Understand  the  relationship  between  the  process  models  and  the  life  cycle  models  (i.e.,  waterfall,  vee, 
and  spiral). 

Tailor  the  SE  processes  in  standards  or  models  to  the  program. 

Identify  the  technical  and  technical  management  processes  of  SE  as  well  as  the  IPTs  that  apply  them. 

Match  key  terms  associated  with  the  SE  processes  to  their  respective  definitions. 

Identify  the  inputs  and  outputs  of  each  process. 

Differentiate  among  various  aspects  of  the  SE  processes  for  each  of  the  following:  application 
consideration  and  IPT  involvement;  architecture  of  a  system;  levels  of  development;  SE  processes 
applied  within  the  acquisition  life  cycle;  and  hardware  versus  software  development/integration. 

Requirements  Development 


K-l 


SYS  101-The  Systems  Engineering  Processes 

Understand  the  purpose,  inputs  and  outputs  of  the  Requirements  Development  process. 

Analyze  capabilities  documents. 

Analyze  lifecycle  impacts  and  their  relationship  to  requirements. 

Identify  all  lifecycle  stakeholders  and  elicit  their  requirements. 

Develop  a  set  of  functional,  physical,  and  operational  requirement  viewpoints  in  accordance  with  SE 
commercial  practices. 

Recognize  different  sources  of  inputs  and  identify  various  requirement  types. 

Distinguish  the  attributes  of  a  well-defined  requirement  and  the  problems  associated  with 
requirements  generation. 

Apply  the  15  requirements  analysis  tasks  to  a  system  at  all  levels  of  development. 

Analyze  requirements  flow  down  and  allocation  into  specifications. 

Analyze  behavioral  requirements  and  design  constraints. 

Focus  on  user  requirements. 

Understand  the  user's  perspective  and  requirements. 

Conduct  requirements  development  and  structure  research  and  development  work  effort  to  match 
user  needs. 

Work  with  the  user  to  conduct  periodic  re-evaluation  of  requirements. 

Develop  requirements  in  an  integrated  framework. 

Requirements  Management 

Understand  the  purpose,  inputs  and  outputs  of  the  Requirements  Management  process. 

Implement  a  disciplined  Requirements  Management  process  from  the  ICD  through  the  system 
requirements  documents  (e.g.,  ICD  to  Request  for  Proposal  (RFP)  performance  requirements  document 
to  contract  specifications  to  product  specifications,). 

Link  lower  level  specifications  to  higher  level  requirements. 

Provide  traceability  from  logical  groupings  (of  functions,  etc.)  to  requirements. 

Formulate  plans  so  as  to  ensure  that  functional,  design,  performance,  environment,  cost  and  schedule 
requirements  are  being  tracked. 
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SYS  101-The  Systems  Engineering  Processes 

For  each  baseline  specification,  implement  program  metrics  to  measure  and  track  the  extent  of 
requirements  changes  in  terms  of  number  of  requirements’  “shall  statements”  changed  from  contract 
award  forward  through  the  development  cycle. 

Institute  a  formal  requirements  traceability  process  that  begins  with  the  ICD  and  evolves  to  include  the 
total  system. 

Document  the  treatment  of  the  user's  interoperability  requirements. 

Require  contractors  to  implement  requirements  tracking  tools  and  flow  down  these  tools  to  the 
subcontractor  base  to  assure  completeness.  When  appropriate,  use  commercially  available  software 
tools  to  implement  the  requirements  management  and  tracking  process. 

Logical  Solution 

Understand  the  purpose,  inputs  and  outputs  of  the  Logical  Solution  process. 

Create  a  behavior  model. 

Develop  a  functional  architecture  during  the  Logical  Solution  process  in  accordance  with  SE  commercial 
practices. 

Distinguish  among  functional  analysis,  allocation,  and  functional  architecture. 

Summarize  tools  used  in  the  Logical  Solution  process  (functional  flow  block  diagram,  timeline 
sheets,  and  requirements  allocation  sheets). 

Understand  the  evolution  of  the  functional  architecture  from  the  established  system  requirements. 
Develop  functional  specifications. 

Design  Solution 

Understand  the  purpose,  inputs  and  outputs  of  the  Design  Solution  process. 

Identify  solution  alternatives  (training,  doctrine,  material,  etc.). 

Analyze  alternative  design  solutions  and  select  one. 

Translate  outputs  of  the  Logical  Solution  process  into  system  product  and  process  designs  which  duly 
consider  life  cycle  issues. 

Analyze  requirements  flow  down  and  allocation  into  specifications. 

Require  completion  of  the  top  level  system  design  and  requirements  allocation  to  subsystems  prior  to 
initiation  of  the  subsystem  design  activity. 

Understand  the  evolution  of  the  physical  architecture. 

Evaluate/  Select  preferred  architecture. 
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SYS  101-The  Systems  Engineering  Processes 

Allocate  functions  to  physical  elements. 

Analyze  industry  standards  to  determine  their  role  in  describing  the  physical  architecture  of  a  system. 
Develop  specification  trees  and  specifications. 

Summarize  the  tools  in  the  Design  Solution  process  (concept  description  sheets  and  schematic  block 
diagrams). 

Develop  a  design  architecture  during  the  Design  Solution  process  in  accordance  with  SE  commercial 
practices. 

Identify  the  inputs  and  outputs  of  the  Design  Solution  process. 

Recognize  how  the  open  systems  design  principles  influence  the  Design  Solution  process. 
Recognize  how  modeling  and  simulation  supports  the  Design  Solution  process. 

Develop  the  training  system  architecture. 

Develop  the  manufacturing  system  architecture. 

Implementation 

Understand  the  purpose,  inputs  and  outputs  of  the  Implementation  process. 

Conduct  make/buy/reuse  analysis. 

Conduct  manufacturing  planning. 

Develop/implement  the  manufacturing  strategy  and  plan. 

Determine  the  major  variables  and  trends  encountered  in  production,  and  how  they  relate  to  other 
functional  areas. 

Manage/execute  acceptance  testing. 

Perform  process  inspection  and  product  acceptance. 

Develop  documentation  for  system  elements. 

Integration 

Understand  the  purpose,  inputs  and  outputs  of  the  Integration  process. 

Understand/develop  individual  system/subsystem  interface  requirements. 

Establish  Interfaces  (software/software,  hardware/software,  human). 
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SYS  101-The  Systems  Engineering  Processes 

Require  systems  level  integration  and  test  planning  early  in  the  program. 

Design  for  the  compatibility  of  all  functional  and  physical  interfaces. 

Define/  refine  interfaces  as  design  matures. 

Conduct  system  hardware  and  software  integration. 

Conduct  system  user  interface  analysis. 

Reinforce  the  importance  of  specifying,  allocating,  and  controlling  interface  requirements  including 
interfaces  among  the  members  of  a  System  of  Systems  (SoS )/  Family  of  Systems  (FoS). 

Integrate,  test  and  modify  the  physical  product  from  lowest  level  through  system  level  (test,  analyze  and 
fix). 

Determine  and  manage  the  impact  of  a  new  system  to  current,  fielded  solutions. 

Verification 

Understand  the  purpose,  inputs  and  outputs  of  the  Verification  process. 

Determine  that  proposed  design  meets  requirements  and  is  achievable  within  existing  programmatic 
constraints. 

Verify  technical  and  operational  performance  against  the  established  baseline  in  accordance  with  policy 
and  legislation  from  the  component  to  the  system  level  throughout  the  life  cycle. 

Apply  the  Verification  process  in  order  to  ensure  the  system  elements  meet  or  exceed  their  requirements. 
Identify  functions  of  the  Verification  process. 

Differentiate  the  methodologies  for  developing,  conducting,  and  analyzing  tests  throughout  the 
systems  acquisition  life  cycle  (e.g.,  verification,  validation,  accreditation  (VV&A),  independent 
verification  and  validation  (IV&V),  metrics,  technical  performance  measurement  (TPM),  and  open 
systems). 

Differentiate  between  the  systems  verification  process  and  mandatory  test  and  evaluation  planning 
requirements. 

Conduct  design  solution  verification. 

Write  a  verification  requirement  for  each  performance  requirement  “shall  statement”  and  incorporate  them 
into  the  requirements  definition/specification  processes.  This  should  take  the  form  of  a  narrative  section 
four  in  the  specification,  (see  JACG  series  of  specification  guides). 

Include  verification  of  interface  requirements  in  development  and  test  plans. 

Identify  deficiencies  that  keep  systems  from  meeting  requirements. 
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Validation 

Understand  the  purpose,  inputs  and  outputs  of  the  Validation  process. 

Perform  requirements  validation. 

Perform  Logical  Solution  and  design  solution  representations  validation. 

Transition 

Understand  the  purpose,  inputs  and  outputs  of  the  Transition  process. 

Define  transportation,  packaging,  handling,  and  storage  requirements. 

Plan  and  monitor  the  fielding  process. 

Develop  a  deployment  system  architecture. 

Technical  Planning 

Understand  the  purpose,  inputs  and  outputs  of  the  Technical  Planning  process. 

Determine  the  sensitivity  of  interrelated  tasks. 

Differentiate  the  role  of  technical  planning  within  the  SE  effort  from  overall  program  planning. 

Distinguish  between  program  planning  and  activities  of  the  Technical  Planning  process. 

Select  control  criteria  and  metrics  for  the  Technical  Planning  process. 

Distinguish  between  event  criteria  planning  and  calendar  time-based  schedule. 

Distinguish  among  the  following  systems  design  influences:  reliability  and  maintainability; 
supportability  and  support;  manufacturing  and  production;  human  systems  integration;  test  and 
evaluation;  and  modeling  and  simulation. 

Recognize  contractor  technical  planning  and/or  execution  efforts. 

Technical  Assessment 

Understand  the  purpose,  inputs  and  outputs  of  the  Technical  Assessment  process. 

Establish  measures  of  performance  and  effectiveness  for  proposed  material  solutions. 

Understand  the  key  analyses  required  to  ensure  that  a  system  meets  customer  needs  and  provide  a 
balanced  set  of  products  and  processes. 

Assess  development  programs  and  identify  risks  and  pitfalls. 
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Assess  contractor  technical  management  (e.g.  engineering,  manufacturing,  and  quality)  systems  and 
processes. 

Decision  Analysis 

Understand  the  purpose,  inputs  and  outputs  of  the  Decision  Analysis  process. 

Apply  quantitative  and  qualitative  tools  to  support  problem  solving  and  decision  making  in  an  acquisition 
environment. 

Understand  group  decision  making  and  voting. 

Determine  risk  and  factor  it  into  the  decision  process. 

Provide  and  document  decision  rationale. 

Risk  Management 

Understand  the  purpose,  inputs  and  outputs  of  the  Risk  Management  process. 

Apply  the  acquisition  risk  management  process  within  an  IPPD  environment  in  order  to  determine  risk. 
Recognize  key  risk  and  risk  management  policy  concepts. 

Distinguish  among  the  risk  management  process  steps  (planning,  assessment,  handling,  and 
monitoring). 

Recognize  the  role  of  risk  management  in  the  DoD  acquisition  process. 

Apply  the  risk  management  process  (plan,  assess,  handle,  and  monitor)  as  a  basis  for  making  sound 
acquisition  program  decisions. 

Define  and  quantify  risk,  and  impacts/benefits  of  risk  management. 

Assess  the  interrelationship  of  technical  risk  with  cost  and  schedule. 

Determine  alternative  approaches  for  moderate/high  risks. 

Develop/utilize  risk  management  methodologies  and  tools. 

Identify  methods,  tools  and  techniques  for  analyzing,  documenting,  and  mitigating/minimizing  ESOH 
risks. 

Establish/utilize  a  risk  reporting  system  and  tracking  techniques. 

Understand  how  risk  is  defined,  identified,  classified  and  quantified. 

Understand  risk,  levels  of  risk,  risk  drivers,  and  constraints. 
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Include  early  simulation  demonstrations  of  the  reuse  candidates  in  the  risk  management  program  as  risk 
mitigation  activities. 

Analyze  risk  management  planning  (prototyping,  spiral  development,  evolutionary  acquisition, 
modifications  and  upgrades). 

Understand  risk  mitigation  techniques  (Non-Developmental  Items  (NDI),  Commercial  Off  the  Shelf 
(COTS),  Technology  Area  Assessment  (TAA),  Technology  Area  Plan  (TAP). 

Reduce  the  risk  of  operational/support  problems. 

Understand  how  hazard  analyses  balance  the  benefits  and  risks  of  both  ESOH  and  system 
performance  considerations. 

Configuration  Management 

Understand  the  purpose,  inputs  and  outputs  of  the  Configuration  Management  (CM)  process. 

Determine  the  role  and  functions  of  the  Configuration  Management  process  in  the  acquisition  process. 

Contrast  the  roles  and  relationships  of  configuration  management,  data  management,  and  interface 
management. 

Understand  the  differences  between  configuration  baselines  versus  program  baselines. 

Generate  configuration  baselines  and  provide  technical  support  for  acquisition  program  baselines. 

Understand  the  configuration  baselines  (functional,  allocated,  and  product),  when  they  are  established, 
and  why  they  are  important. 

Perform  baseline  management  including  audits. 

Understand  the  four  functions  of  Configuration  Management  and  their  inter-relationships. 
Understand/execute  the  change  process. 

Understand  the  three  different  kinds  of  changes  within  the  configuration  control  function. 

Track,  approve  and  control  engineering  changes. 

Understand  the  internal  operations  of  a  Configuration  Control  Board  (CCB). 

Distinguish  among  CM,  CM  benefits,  CM  reference  documents,  and  CM  planning,  and  also  identify  the 
four  major  sub  processes  of  CM. 

Differentiate  the  following  configuration  identification  function  elements:  configuration  items; 
configuration  documentation;  configuration  baselines;  and  program-unique  specifications. 

Apply  the  following  configuration  control  function  elements:  Engineering  Change  Proposals  (ECP); 
Request  for  Deviation  (RFD);  and  CCBs. 
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Differentiate  between  the  configuration  status  accounting  and  configuration  audit  functions. 

Manage  and  communicate  changes  to  systems  in  all  phases  of  the  life  cycle. 

Data  Management 

Understand  the  purpose,  inputs  and  outputs  of  the  Data  Management  process. 

Contrast  the  roles  and  relationships  of  configuration  management  (CM),  data  management,  and  interface 
management. 

Define  the  minimum  essential  life  cycle  data  requirements. 

Obtain  technical  data  and  trace  data  to  requirements. 

Understand  the  relationship  of  the  Contract  Data  Requirements  List  (CDRL)  to  the  Data  Item  Description 
(DID)  and  RFP. 

Ensure  the  quality  of  the  data. 

Establish/maintain  data  management  database. 

Work  in  an  integrated  data  environment. 

Interface  Management 

Understand  the  purpose,  inputs  and  outputs  of  the  Interface  Management  process. 

Differentiate  the  following  interface  management  function  elements:  Interface  Control  Working  Groups 
(ICWG)  and  interface  control  documentation  (ICD). 

Ensure  the  integrity  of  interface  controls. 

Design  Considerations 

Open  Systems  Architecture 

Understand  open  systems  architectures,  disciplines,  tools  and  methods  and  application  to  system 
interoperability. 

Document  technical  approach  to  using  an  open  systems  approach  to  system  design. 

Promote  use  of  contractor/government-off-the-shelf  NDIs. 

Assess  the  use  of  NDI  including  their  constraints,  advantages,  and  disadvantages. 

Architectures  and  Interoperability 

Determine  the  purpose  and  timing  of  architectures  over  the  system  life  cycle. 
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Understand/assess  standards  and  interoperability  requirements. 

Define  systems  of  systems  and  understand  their  complex  organization  and  management. 

Ensure  that  the  proposed  system  functionally  operates  with  other  systems,  units,  or  forces,  to 
include  U.S.  or  other  coalition  partners. 

Synthesize  multiple  architectures. 

Conduct  system  architecture  modeling  and  analysis. 

Develop  systems  using  international  interoperability  standards. 

Understand  the  increased  emphasis  of  interoperability  as  a  Key  Performance  Parameter  (KPP)  and 
ensure  it  is  reflected  in  the  solicitation. 

Identify  and  describe  basic  principles  of  technical  standards  as  they  relate  to  system  development 
and  interoperability. 

Comply  with  Joint  Technical  Architecture  (JTA)  requirements. 

Understand  and  apply  JTA  requirements  and  standards. 

Comply  with  Defense  Information  Infrastructure  Common  Operating  Environment  (Dll  COE). 

Perform  an  interoperability  performance  analysis. 

Perform  analysis  to  identify  linkages,  connections,  processes  and  delay  time  that  affect 
interoperability. 

Understand  framework  to  look  at  interoperability  through  layers  such  as  process,  software, 
information  and  influences. 

Participate  in  developing  integrated  architectures  for  DoD  systems  and  understand  the  interoperability 
certification  process. 

Software 

Recognize  the  complexity  of  software  development,  its  integral  nature  to  the  SE /  software  system  safety 
processes,  and  top-level  "best  practices"  for  successful  software  development. 

Define  software  requirements. 

Understand  the  characteristics  of  a  well-defined  software  requirement. 

Develop  software  functionality  specifications. 

Demonstrate  "best  practices"  for  the  acquisition  of  a  software  intensive  system. 

Understand  the  similarities  and  differences  between  hardware  and  software  development. 
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Determine  the  key  processes  that  should  be  used  by  developers  to  create  quality  software  products. 
Know  software  development  principles  and  techniques. 

Develop  evaluation  and  assessment  criteria  to  measure  software  progress. 

Apply  newly  developed  software  evaluation  tools. 

Use  state  of  the  art  tools  and  techniques  available  for  planning,  measuring  and  predicting  software 
development  progress. 

Commercial  Off  the  Shelf  Items  and  Non  Developmental  Items 

Know  and  understand  benefits  and  opportunities  of  using  /  transitioning  to  commercial  items  where 
available. 

Understand  the  appropriate  documentation  and  potential  ESOH  implications  during  COTS  selection. 
Understand  NDI  &  COTS  use  to  meet  technical  goals. 

Conduct  SE  analyses  of  the  reuse  and  COTS  candidates  to  determine  that  the  candidates  exist  in 
acceptable  usable  form,  that  they  meet  the  new  system  requirements,  and  that  they  are  well  documented 
and  designed  for  reuse. 

Develop  appropriate  T&E  strategy  for  commercial  items. 

Manufacturing  Capability  and  Producibility 

Understand  the  producibility  concepts  and  activities  throughout  the  acquisition  life  cycle  phases. 

Recognize  the  major  producibility  goals  of  the  design  effort  and  the  DoD  quality  process  which  translates 
a  released  design  to  a  producible  product. 

Determine  the  role  of  manufacturing  considerations  in  the  SE  processes  throughout  the  acquisition  life 
cycle. 

Evaluate  alternative  /new  manufacturing  processes  and  capability. 

Understand  industry/commercial  approaches  to  manufacturing  innovation,  i.e.  conversion  and 
reconstitution. 

Understand  manufacturing  technology. 

Conduct  producibility  studies. 

Understand  producibility  issues  and  how  to  manage  the  design  for  producibility. 

Understand  environmental  trade-offs  between  basic  manufacturing  techniques. 
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Demonstrate  that  all  critical  manufacturing  processes  are  under  statistical  control  and  consistently 
producing  items  within  the  quality  standards  and  tolerances  for  the  overall  product  before  production 
begins. 

Quality  Assurance 

Plan  and  operate  quality  assurance  and  baseline  measurement. 

Develop  system  quality  requirements. 

Conduct  quality  assurance. 

Reliability,  Availability  and  Maintainability 

Conduct  system  reliability,  availability,  and  maintainability  analysis. 

Determine  the  system  maintenance  concept  and  conduct  maintenance  forecasting  and  planning. 
Understand  the  levels  of  maintenance  and  how  they  affect  life  cycle  costs. 

Design  for  reliability,  availability,  and  maintainability. 

Demonstrate  product  reliability  before  the  start  of  production  by  testing  to  identify  the  problems,  make 
design  corrections,  and  retest  the  new  design. 

Understand  the  most  common  methods  used  to  measure  reliability  and  maintainability. 

Understand  the  differences  and  similarities  between  hardware  and  software  reliability  and  maintainability. 
Identify  the  impact  of  reliability,  availability,  maintainability  on  system  support  and  total  ownership  costs. 
Acquisition  Logistics  and  Supportability 

Determine  how  acquisition  logistics  activities  impact  and  relate  with  other  functional  areas  within  the 
acquisition  life  cycle. 

Determine  the  acquisition  logistics  support  activities  and  requirements  associated  with  fielding/ 
deployment  and  post  production  support  of  a  system. 

Prepare  logistics  planning  and  execute  into  technical  effort. 

Recognize  the  importance  of  supportability  to  achieving  the  system  readiness  requirements  and  reducing 
life-cycle  costs. 

Conduct  logistics  integration. 

Understand  designing  for  supportability. 

Develop  or  evaluate  design  changes  in  response  to  supportability  issues. 
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Know  and  understand  supportability  analysis  tools  and  techniques. 

Recognize  the  importance  of  the  10  support  elements  in  supportability  planning. 

Make  critical  decisions  concerning  system  supportability. 

Understand  the  effects  technology  has  on  supportability. 

Develop  training  for  field  users. 

Prepare  logistics  planning  and  execute  into  technical  effort. 

Conduct  supply  support  planning  and  develop  system  supply,  support,  and  spares  management. 

Assess  manpower  and  personnel  impacts. 

Human  Systems  Integration 

Understand  human  and  cognitive  factors  and  their  affect  on  the  system  technical  effort. 

Develop  Human  System  Integration  (HSI)  requirements. 

Design  for  usability,  human  factors,  and  human  systems  integration. 

Environmental,  Safety,  and  Occupational  Health 
Integrate  ESOH  considerations  in  systems  acquisition. 

Describe  ESOH  statutes,  regulations,  policies,  international  agreements,  and  government/  industry 
standards  and  how  they  affect  systems  acquisition  programs  throughout  the  life-cycle. 

Identify  and  describe  key  ESOH  functional  areas  that  should  be  evaluated  for  potential  ESOH  risks  to  the 
program  and  to  the  users  throughout  the  systems'  life-cycle:  e.g.  National  Environmental  Policy  Act  / 
Executive  Order  12114;  hazardous  materials  management/  pollution  prevention;  and  safety  and  health,  to 
include  explosives  safety/insensitive  munitions,  system  safety,  personnel  safety/occupational  health 

Identify  and  describe  the  types  of  ESOH  information  required  in  key  acquisition  documents  in  accordance 
with  DoD  systems  acquisition  policy;  key  documents  include:  RFPs,  Contracts,  and  Government 
Statements  of  Work  (SOWs);  ICD;  ODD;  CPD;  AoA;  Acquisition  Program  Baseline  (APB);  acquisition 
strategy;  Test  and  Evaluation  Master  Plan  (TEMP);  Cost  Analysis  Requirements  Document  (CARD); 
Programmatic  ESOH  Evaluation;  SEP;  HSI  plan;  Logistics  Support  Analysis  (LSA)/lntegrated  Logistics 
Support  Plan  (ILSP);  demilitarization/disposal  plans. 

Conduct  ESOH  evaluations  as  delineated  in  DoDI  5000.2. 

Provide  detailed  methods  for  ESOH  and  National  Environmental  Policy  Act  /Executive  Order  12114 
compliance. 

Provide  detailed  tools  and  methods  to  implement  system  safety,  MIL-STD  882D,  and  occupational 
health  considerations. 


K-13 


SYS  101-The  Systems  Engineering  Processes 

Describe  the  process  for  identifying  hazardous  materials  management,  and  the  processes  for 
selecting  alternatives  for  use  on  the  system. 

Provide  methods  to  implement  pollution  prevention  in  the  program. 

Understand  system  safety  principles,  including  identifying,  selecting,  and  evaluating  source 
materials  and  processes. 

Assess  the  impacts  of  ESOH  considerations  in  the  SE  processes. 

Apply  historical  lessons  learned  from  legacy  systems  to  future  systems. 

Survivability 

Develop  survivability  requirements. 

Corrosion  Prevention  and  Control 

Conduct  corrosion  prevention  and  control  planning  to  derive  system  requirements. 

Employ  material  selection,  packaging,  and  maintenance  procedure  decisions  conducive  to  corrosion 
prevention  and  control. 

Describe  the  facilities  engineering  process  and  how  it  relates  to  the  DoD  Acquisition  Process;  describe 
how  a  program  manager  is  made  aware  of  facilities  engineering  issues;  describe  how  facilities 
engineering  issues  affect  acquisition  programs;  and  describe  the  funding  process  for  facilities 
engineering. 

Disposal  and  Demilitarization 

Understand  the  goals  and  objectives  of  disposal  and  demilitarization  and  their  impact  on  the  SE 
processes  during  the  development  of  a  weapon  system. 

Information  Assurance 

Assess  key  information  assurance  requirements  that  characterize  the  current  DoD  acquisition 
environment. 

Incorporate  information  assurance  requirements  into  system  design  activities  to  ensure  availability, 
integrity,  confidentiality,  authentication  and  non-repudiation  of  critical  system  information. 

System  Security 

Conduct  system  security  engineering  and  derive  system  requirements. 

Test  and  Evaluation 

Determine  the  role  of  T&E  in  the  systems  engineering  and  acquisition  management  processes. 
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Determine  the  fundamental  role  of  Developmental  Test  and  Evaluation  (DT&E)  in  the  acquisition  life 
cycle. 

Include  test  and  evaluation  as  a  topic  in  program  and  design  reviews. 

Assure  that  there  is  only  one  test  requirements  baseline  for  the  program  that  all  stakeholders  use. 

Understand  the  differences  between  DT&E  and  Operational  Test  and  Evaluation  (OT&E)  and  the  various 
tests  associated  with  the  two  types. 

Provide  requirements  for  test  management  and  diagnostic  equipment  and  ground  support  equipment. 
Plan  appropriate  T&E  of  commercial  items  and  NDI. 

Recognize: 

--The  different  types  of  T&E. 

--The  organizations  responsible  for  each. 

--The  reason  for  heavy  DoD  commitment  to  T&E. 

--T&E  planning  and  its  function  as  the  essential  feedback  mechanism  for  the  SE  processes. 

Sustainment 

Include  sustainment  as  a  topic  throughout  the  design  process;  include  sustainment  specialists  in  program 
planning  activities. 

Systems  Engineering  Techniques  and  Tools 

Understand  SE  tools  and  methods,  e.g.,  TPM,  IDEF,  Statistical  Process  Control,  Quality  Function 
Deployment,  N2,  Pareto  and  other  charts,  influence  diagrams,  critical  path  analysis. 

Understand  Functional  Flow  Block  Diagrams  (FFBDs),  timeline  sheets,  and  requirements  allocation 
sheets. 

Understand  the  documentation  tools  available  (e.g.  concept  description  sheet  and  schematic  block 
diagrams). 

Understand  design  technologies  and  tools  to  include:  Computer  Aided  Design  (CAD),  Computer  Aided 
Engineering  (CAE),  Computer  Aided  Manufacturing  (CAM),  and  Universal  Modeling  Language  (UML). 

Understand  the  activities,  tools  and  documentation  for  integrating  ESOH  considerations  into  the  SE 
processes. 

Systems  Engineering  Plan 
Develop/coordinate  and  implement  the  SEP. 

Relate  the  similarities  and  differences  between  the  government  and  contractor  SEP. 
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Reflect  acquisition  strategy  in  government  SEP. 

Understand  how  program  political  and  budget  issues  may  influence  the  SEP. 

Establish  a  SEP  early  in  the  program  definition,  and  update  it  continuously  as  the  program  matures, 
through  system  operations  and  support. 

Document  in  the  SEP  a  description  of  the  systems  engineering  integration  on  program  IPTs 
including  resources,  staffing,  management  metrics  and  integration  mechanisms. 

Describe  how  the  processes  will  be  implemented  and  how  they  will  be  tailored  to  meet  individual 
acquisition  phase  objectives. 

Describe  how  the  SE  processes  will  support  the  technical  and  programmatic  products  required  of 
each  phase. 

Describe  how  the  technical  baseline  will  be  developed,  managed,  and  used  to  control  system 
requirements,  design,  integration,  verification,  and  validation. 

Describe  metrics  (e.g.,  technical  performance  measures)  for  the  technical  effort  and  how  these 
metrics  will  be  used  to  measure  progress. 

Describe  how  systems  engineering  activities  will  be  integrated  within  and  coordinated  across  IPTs; 
how  the  IPTs  will  be  organized;  what  SE  tools  they  will  employ;  and  their  resources,  staffing, 
management  metrics,  and  integration  mechanisms. 

Describe  how  SE  processes  are  integrated  in  the  program's  overall  integrated  schedules. 

Work  Breakdown  Structure 

Recognize  the  inherent  power  of  a  well-designed  Work  Breakdown  Structure  (WBS)  and  its  application 
throughout  the  SE  processes. 

Relate  the  physical  or  system  architecture  and  specifications  to  the  WBS. 

Develop  a  WBS  based  on  a  given  physical  architecture. 

Distinguish  among  three  uses  of  the  WBS  (organizational,  business,  and  technical)  and  be  able  to 
recognize  various  types  of  work  breakdown  structures. 

Understand  the  derivation  and  multiple  uses  (e.g.  Cost  Performance  Reporting  (CPR)  system,  work 
package  relationship,  SOW  relationship,  and  the  primary  functions  utilization)  of  the  WBS. 

Recognize  how  the  IPTs  are  organized  using  the  WBS. 

Recognize  how  the  design  (or  physical)  architecture,  specification  tree,  and  system  architecture 
support  the  WBS. 

Integrated  Master  Plan/Schedules 
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Require  that  the  Integrated  Master  Plan  (IMP)  has  well-defined  accomplishments  with  entrance  and  exit 
criteria  for  program  milestones  and  events. 

Ensure  that  the  Systems  Requirement  Review  (SRR)  is  a  key  event  in  the  IMP  with  exit  criteria  that 
assures  the  system  specification  is  complete,  including  a  complete  set  of  verification  requirements. 

Map  the  SE  processes  defined  in  the  program  SEP  to  the  IMP,  including  entrance  and  exit  criteria  for  the 
program  reviews  and  events.  Specifically  identify  SE  task  completions  as  entrance  and  exit  criteria. 

Evaluate  the  IMP  and  Integrated  Master  Schedule  (IMS)  for  T&E  planning  realism  during  the  source 
selection. 

Require  the  offeror  to  propose  a  system  level  IMP  and  IMS.  Evaluate  these  products  during  source 
selection  to  ensure  that  the  key  SE  and  engineering  development  events  have  appropriate  entry  level 
criteria  and  that  the  planned  SE  task  durations  are  adequate. 

Keep  the  system  IMP/IMS  at  the  appropriate  level  by  focusing  on  key  gating  events  and  important  criteria. 

Establish  subsystem  IMP/IMS  as  appropriate  and  flow  the  requirement  to  subcontractors  and  vendors  as 
appropriate. 

Enforce  the  criteria  included  in  the  IMP;  don’t  gloss  over  event  and  accomplishment  exit  criteria. 

When  actions  are  needed  to  close  criteria  partially  met,  establish  near  term  follow-up  plans  to  assure  the 
actions  are  completed  and  the  IMP  criteria  are  fully  met. 

Value  Engineering 

Know,  understand  and  be  able  to  use  value  engineering  methods  and  tools. 

Technical  Performance  Measurement 
Define  effectiveness  measures. 

Determine  the  role  of  TPMs  in  the  SE  processes. 

Determine  how  TPMs  will  be  captured  in  the  SEP. 

Understand  how  technical  parameters/TPMs  are  applied  to  performance-based  progress  measurement  in 
SE. 

Understand  performance  based  management. 

Compare  performance  achievements  to  date  with  plan. 

Develop  new/modified  performance  goals. 

Develop  recommended  approach  for  TPM. 

Manage  technical  performance  measure  variances. 
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Understand  the  conceptual  basis  and  relevant  terms  for  TPM  charts  and  how  they  are  linked  to  critical 
Measures  of  Performance  (MOP). 

Develop  TPMs  when  given  a  set  of  system  requirement  descriptions. 

Recognize  the  performance  measurement  requirements. 

Understand  relevant  terms  identified  within  TPM  charts. 

Analyze  a  TPM  chart  to  determine  the  necessary  management  action. 

Determine  criteria  and  timing  for  selecting  TPMs. 

Know  and  understand  use  of  TPMs  and  their  impact  on  cost  and  ability  to  meet  contract  technical 
requirements. 

Trade  Studies 

Conduct  trade  studies  using  system  simulations  where  appropriate  and  document  results. 

Understand  the  key  alternatives  or  tradeoffs  made  as  the  system  evolves  from  the  conceptual  level  of 
development  to  the  detailed  subsystem  level  of  development. 

Understand  lifecycle  trade  study  techniques  and  methods. 

Understand  the  role  of  trade  studies  in  the  development  of  functional  and  performance  requirements. 

Propose  a  trade  study  methodology,  conduct  an  analysis  in  accordance  with  SE  commercial  practices. 

Understand  where,  when,  and  how  (methodology)  trade  studies  are  used  within  the  SE  processes 
to  resolve  conflicting  requirements. 

Recognize  how  trade  studies  balance  system  life  cycle  concerns. 

Recognize  the  role  of  effectiveness  analyses  in  support  of  trade  studies. 

Modeling  and  Simulation 

Use  Modeling  and  Simulation  (M&S)  throughout  the  life  cycle  of  a  system  and  assess  the  contractor's  use 
of  M&S. 

Apply  M&S  to  identify  and  simulate  design  issues  and  risks. 

Understand  and  determine  how  to  apply  M&S  when  conducting  performance  studies,  effectiveness 
studies,  tradeoff  analysis,  risk  analysis,  sensitivity  analysis  and  cost  analysis. 

Plan  and  accomplish  early  simulation  demonstrations,  prior  to  Preliminary  Design  Review  (PDR),  of  the 
reuse  candidates  on  system  simulations,  i.e. ,  verify  the  reusability  early. 

Be  capable  of  using  and  understanding  the  basic  tenets  of  M&S. 
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Know  and  understand  the  types  of  models  (physical,  mathematical,  logical)  and  the  common  pitfalls  and 
limitations. 

Understand  the  models  and  simulations  associated  with  requirements  generation,  program  management, 
design  and  engineering,  manufacturing,  T&E,  logistics  support  and  training. 

Identify  and  assess  the  M&S  requirements,  benefits,  pitfalls,  planning  and  applications  in  systems 
acquisition. 

Understand  system  dynamics,  discrete  event  models,  discrete  multi-variate  modeling. 

Prototyping 

Understand  the  role  of  prototyping  in  spiral  development. 

Incorporate  early  prototyping. 

Manage  experimentation  and  prototyping. 

Robust  Engineering 

Respond  quickly  and  effectively  to  changing  conditions  or  events  that  impact  the  program's  SE 
processes. 

In  initial  stages  of  product  development,  instill  flexibility  in  both  resources  provided  and  the  product’s 
performance  requirements  to  allow  for  the  uncertainties  of  technical  progress. 

In  initial  stages  of  product  development,  instill  high  standards  for  technical  maturity. 

Understand  and  use  robust  design  techniques  and  tools  for  the  system. 

Ensure  process  variability  reduction  implementation. 

Modular  Open  System  Design 

Employ  modular  design. 

Designate  key  interfaces. 

Design  for  affordable  change. 

Develop  an  integrated  roadmap  for  design  and  development. 

Use  open  standards. 

Assess  feasibility  of  using  widely-supported  standards. 

Test  and  Evaluation  Master  Plan 
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Understand  the  Test  &  Evaluation  Master  Plan  (TEMP),  and  other  related  documents  and  their 
relationship  to  the  SEP  and  the  acquisition  process. 

Coordinate  in  the  development  of  the  TEMP. 

Technical  Data  Packages 

Prepare  technical  data  packages. 

Determine  the  purpose  and  timing  of  technical  data  packages  and  other  system  specific  information  over 
the  lifecycle. 

Specifications 

Determine  the  purpose  and  timing  of  specifications  over  the  lifecycle. 

Recognize  the  content  and  format  of  a  specification. 

Employ  templates  for  performance  specifications;  include  examples  such  as  those  contained  in  the 
JACG’s  series  of  specification  guides. 

Earned  Value  Management 

Understand  Earned  Value  (EV)  principles,  evaluate  EV  data,  and  make  recommendations. 

Recognize  the  value  and  benefits  of  EVM  in  the  acquisition  process. 

Analyze  the  contractor's  status  by  applying  EV  analysis  techniques. 

Ensure  that  contractors  have  an  EVM  system  that  reports  cost  and  schedule  information  at  a  level  of  work 
that  provides  information  specific  to  software  development. 

Technical  Reviews 

Understand  how  the  Program  Manager  (PM)  and  SE  staff  should  prepare  for  and  follow  up  on  technical 
reviews. 

Understand  how  each  configuration  baseline  and  applicable  specifications  relate  to  specific  technical 
reviews. 

Develop  plans  for  technical  reviews  throughout  the  life  cycle. 

Identify  the  level/type  of  the  review. 

Understand  the  role  of  technical  reviews  as  another  means  of  controlling  technical  risk  and  providing 
performance-based  progress  measurement. 

Establish  exit/entry  criteria  and  do  not  close  a  technical  review  until  all  established  exit  criteria  are  met. 
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Recognize  the  purpose  of  technical  reviews  as  a  means  of  controlling  technical  risk  and  assessing  design 
maturity. 

Differentiate  how  each  configuration  baseline  corresponds  to  an  applicable  program-unique 
specification(s)  and  a  specific  technical  review. 
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These  duties  and  tasks  are  divided  by  shaded  headings.  The  major  headings  are 
highlighted  in  dark  gray:  IPPD,  Acquisition,  Technical  Reviews,  and  SE  by  Phases.  The 
sub-headings  are  shown  in  lighter  gray. 

SYS  201 -Systems  Engineering  by  Acquisition  Phase 

Assess  key  policies,  laws,  and  regulations;  support  issues;  development  paradigms  and  strategies  that 
characterize  the  current  DoD  acquisition  environment. 

IPPD 

Understand  the  implementation,  barriers,  organization  structure,  etc.  of  the  IPPD/IPT  concept. 

Select  organization  and  teaming  techniques,  i.e.  IPT  structure. 

Encourage  early  and  strong  government/industry  SE  activity  prior  to  the  forming  and  chartering  of  specific 
IPTs. 

Designate  as  IPTs  only  those  teams  that  will  have  the  day-to-day  responsibility  for  developing  and 
delivering  a  product  and  the  expertise  to  do  so. 

Assign  responsibilities  for  ESOH  management  and  establish  ESOH  IPTs/working  groups. 

Recognize  the  nature  of  group  interaction,  its  principle  phases,  and  how  teams  arrive  at  better  solutions. 

Apply  the  IPPD  principles  within  the  SE  processes  as  performed  by  IPTs. 

Identify  the  key  IPPD  concepts  and  implementation  strategies. 

Associate  the  following  IPT  concepts  with  their  definitions:  IPT  Principles;  IPT  roles  and 
responsibilities;  IPT  organizational  structure;  team  dynamics  and  government’s  role;  team  metrics; 
team  size;  team  leader;  IPT  charter;  IPPD  tools. 

Distinguish  SE  from  IPPD. 

Develop  and  implement  an  organizational  structure  where  SE  spans  the  entire  organization,  rather  than 
separating  SE  into  a  specific  IPT. 

Prepare  budgets  for  SE  IPTs.. 

Identify  ESOH  resource  requirements. 

Evaluate  organization,  communication  and  teaming  techniques  that  facilitate  IPPD. 

Understand  the  role  of  government  oversight  when  participating  on  contractor's  IPTs. 

Acquisition 
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Assess  the  interaction  of  SPRDE  with  the  CJCS  3170.01  JCIDS  process,  the  DoD  Planning, 
Programming,  Budgeting  and  Execution  (PPBE)  process,  and  the  DoD  5000  series  acquisition  life  cycle 
process. 

Operate  in  a  multi-service  environment. 

Apply  the  appropriate  defense  acquisition  policies  and  SE  processes  necessary  for  successful  execution 
of  phase  activities. 

Recognize  the  need  fora  phased  acquisition  approach  and  a  tailored  acquisition  strategy. 

Understand  roles/responsibilities  of  acquisition  stakeholders. 

Evolutionary  Acquisition 

Develop  EA  design  strategies  and  ensure  the  system  design  supports  the  EA  approach. 

Match  evolutionary  requirements  with  capability  needs. 

Use  evolutionary  acquisition  and  spiral  development  as  a  framework  for  system  and  software 
development  programs. 

Establish  requirements  early  in  the  program  to  accomplish  technology  refresh  to  preclude  obsolescence 
early  in  the  program  and  include  the  topic  throughout  the  development  process. 

Perform  the  expectancy  analyses  for  parts  susceptible  to  early  obsolescence  as  part  of  the  SE  processes 
very  early  in  the  development  cycle  and  include  design  considerations  that  facilitate  technology  refresh. 

Use  the  SE  processes  to  balance  program  risk,  system  performance,  and  cost  over  the  lifecycle  of  a 
system  incorporating  an  evolutionary  acquisition  approach. 

Establish  program  plans  to  accommodate  requirements  changes  that  exceed  the  program  baseline  by 
deferring  them  to  future  upgrades,  modifications,  or  increments. 

Address  requirements  identified  subsequent  to  the  System  Requirements  Review  (SRR)  in  future 
upgrades. 

Technical  Reviews 

Understand  the  objective  of  specific  major  technical  reviews,  issues  to  be  addressed  in  each,  and  their 
relationship  to  the  acquisition  life  cycle. 

Associate  the  types  of  major  reviews  with  the  issues  to  be  addressed  in  each,  and  each  review’s 
relationship  to  the  acquisition  life  cycle. 

Conduct  technical  reviews  to  provide  the  PM  with  an  integrated  assessment  of  program  technical  risk  and 
readiness  to  proceed  to  the  next  technical  phase  of  effort. 

Form  a  team  of  independent  SME  and  program  team  representatives  to  perform  technical  reviews. 

Appoint  an  independent  technical  authority  to  chair  the  technical  review  team. 


K-23 


SYS  201 -Systems  Engineering  by  Acquisition  Phase 

Include  technical  reviews  in  contractual  documents. 

Tailor  the  technical  reviews  in  conjunction  with  the  SE  process  to  fit  the  acquisition  program. 
Approve  technical  baselines  at  technical  reviews. 

Conduct  technical  reviews  using  an  event  driven  approach  rather  than  by  schedule. 

Develop  metrics  for  use  in  technical  reviews. 

Identify  potential  problems  and  propose  solutions  during  technical  reviews. 

Develop  and  defend  a  checklist  based  on  the  most  relevant  event  criteria  for  a  specific  technical  review. 
SE  by  Phases 

Address  SE  upfront  and  early  (all  acquisition  phases  not  just  System  Development  and  Demonstration 
(SDD). 

Explain  the  interrelationships  between  different  life  cycle  activities  including  prerequisite  activities  and 
cause  and  effect  between  phases. 

Understand  the  inputs  and  outputs  to  each  acquisition  phase. 

Understand  the  systems  acquisition  life  cycle  phases  and  the  major  activities  to  be  accomplished  in  each 
phase. 

Incorporate  ESOH  consideration  throughout  the  system's  life-cycle. 

Emphasize  life-cycle  cost  implications  in  all  program  management  phases  and  decisions. 

Quantify  milestone  requirements. 

Prepare  reports  for  milestone  decision  reviews. 

Establish  exit  criteria  to  enter  subsequent  phase. 

Solicitation  and  Source  Selection 

Require  in  section  L  of  the  solicitation,  an  SE  process  description  tailored  to  the  program  as  part  of  the 
proposal.  This  could  be  in  a  contractor  format,  but  must  address  the  SE  processes  as  they  will  be  applied 
to  the  program. 

Explain  how  the  SOW  and  RFP  are  evaluated  from  an  SE  perspective. 

Develop  performance-based  work  statements  or  statements  of  objectives  (SOO). 

Contrast  the  purpose,  preparation,  and  evaluation  strategies  of  the  SOO  with  those  of  the  SOW. 

Prepare  a  SOW  and  be  able  to  critique  the  structure  and  content  of  one. 
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Recognize  how  the  acquisition  excellence  and  source  demands  impact  solicitation  development. 

Recognize  the  interrelationships  between  the  acquisition  management  systems  data  list  (AMSDL), 
the  CDRL,  and  the  DID. 

Recognize  how  evaluation  factors  in  the  RFP  are  utilized  during  the  source  selection  process  to  identify 
SE  requirements. 

Identify  the  ESOH  considerations  that  should  be  included  in  the  RFP,  SOO,  and  SOW. 

Require  proposed  reuse  to  be  fully  supported  in  the  proposals  and  consider  risks  in  the  evaluation.  The 
proposal  solicitation  should  include  a  reuse  checklist  to  address  the  soundness  of  the  proposed  reuse 
candidates  and  approach,  confirm  reuse  compliance  with  requirements,  quantify  the  attendant  risk,  and 
require  development  of  risk  mitigation  plans. 

Establish  a  requirement  in  proposal  solicitations  for  delivery  of  a  preliminary  specification  tree  and 
description  of  the  SE  processes  that  will  be  used  to  maintain  the  specifications  and  specification  tree  over 
the  life  of  the  program  in  the  proposal. 

Understand  the  basic  ethical  issues  related  to  government  employees  and  professional  engineers  in 
source  selection. 

Prepare  technical  aspects  of  the  source  selection  plan. 

Develop  technical  evaluation  criteria  for  the  technical  proposal. 

Evaluate  technical  proposals. 

Nominate  SE  representatives  to  the  Source  Selection  Evaluation  Board. 

Employ/develop  sourcing  strategies  that  emphasize  best  value. 

Concept  Refinement 

Develop  the  various  SE  related  inputs  and  outputs  of  the  Concept  Refinement  phase. 

Analyze  what  is  achievable  within  the  cost,  schedule,  and  technical  envelope. 

Determine  how  the  TEMP  is  used  to  integrate  T&E  planning  activities  in  support  of  a  program's 
acquisition  strategy. 

Understand  the  purpose  of  Advanced  Technology  Demonstrations. 

Conduct  market  research. 

Know  and  understand  future  technological  advances  that  can  be  incorporated  into  system  development 
programs. 

Assess  technology  opportunities  and  evaluate  the  feasibility,  maturity,  and  risk. 
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Apply  the  science  and  technology  base  to  solve  military  problems  and  create  opportunities  and  options. 

Evaluate  the  effective  execution  of  the  entire  Concept  Refinement  phase  using  the  SE  processes. 

Analyze  technology  opportunities  and  their  impact  on  feasibility,  maturity,  safety,  risk,  affordability  and 
utility  tradeoffs  to  provide  future  military  capabilities. 

Perform  Concept  Refinement  decomposition  and  definition  activities. 

Interpret  user  needs;  Analyze  operational  capabilities  and  define  environmental  constraints. 

Develop  concept  performance  (and  constraints)  definition  and  verification  objectives. 

Decompose  concept  performance  into  functional  definition  and  verification  objectives. 

Decompose  concept  functional  definition  into  component  concepts  and  assessment  objectives. 

Develop  component  concepts,  including  enabling/critical  technologies;  Update  constraints  and 
cost/risk  drivers. 

Perform  Concept  Refinement  integration  and  verification/validation  activities. 

Perform  requirements  validation. 

Assess/analyze  enabling/critical  components  versus  capabilities. 

Assess/analyze  concept  system  versus  functional  capabilities. 

Assess/analyze  concept  and  verify  the  system  concept’s  performance. 

Analyze  and  assess  concepts  versus  defined  user  needs  and  specified  environmental  constraints. 
Prepare  products/outputs  of  the  SE  process  in  Concept  Refinement. 

Conduct  AoA. 

Develop  a  Technology  Development  Strategy  (TDS). 

Develop  a  SEP. 

Choose  a  Preferred  System  Concept  (PSC). 

Consider  technology  issues. 

Develop  a  T&E  strategy. 

Create  exit  criteria. 

Conduct  economic  analysis  (Major  Automated  Information  System  (MAIS)  only). 
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Conduct  Concept  Refinement  technical  reviews. 

Prepare  for  and  conduct  Initial  Technical  Review  (ITR). 

Prepare  for  and  conduct  Alternative  System  Review  (ASR). 

Identify  the  ESOH  considerations  that  should  be  included  in  the  ICD. 

Technology  Development 

Develop  the  various  SE  related  inputs  and  outputs  of  the  Technology  Development  phase. 

Include  basic  SE  on  programs  that  have  substantial  requirements  definition  and  top  level  efforts  prior  to 
milestone  B,  e.g.,  during  Technology  Development.  Complete  the  system  level  requirements  and  design 
per  defined  criteria  prior  to  Milestone  (MS)  B  if  appropriate. 

Recognize  the  state  of  U.S.  technology,  the  role  and  planned  evolution  of  science  and  technology,  and 
how  these  two  elements  apply  to  the  different  phases  of  defense  acquisition. 

Understand  technology  transfer,  transition,  and  insertion. 

Construct  the  technology  development  cycle;  describe  the  purpose  of  each  phase  of  the  cycle,  and 
explain  how  it  relates  to  the  acquisition  life  cycle. 

Determine  which  phase  in  the  technology  development  cycle  is  involved  when  inserting  technology. 

Recognize  how  technology  development  programs  mature  across  the  technology  development 
cycle. 

Understand  the  effects  technology  has  on  the  following:  manufacturing  technology;  industrial 
capabilities;  technology  support;  and  non-developmental  items  and  commercial  items. 

Recognize  the  role  of  an  open  systems  approach  within  technology  development  and  insertion. 

Understand  current  state-of-the-art  and  mechanisms  to  introduce  technology. 

Conduct  market  research. 

Accurately  assess  and  understand  the  risks  associated  with  current  technology  maturity  in  relation  to 
program  needs. 

Assess  technological  opportunities  and  evaluate  the  feasibility,  maturity  and  risk. 

Evaluate  technology  maturation  to  support  short  cycle  time  in  system  development. 

Evaluate  the  effective  execution  of  the  entire  Technology  Development  phase  using  the  SE  processes. 

Determine  the  technical  management  activities  that  must  be  completed  prior  to  transitioning  a  program 
into  system  acquisition. 
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Perform  Technology  Development  decomposition  and  definition  activities. 

Interpret  user  needs;  Analyze  operational  capabilities  and  define  environmental  constraints. 

Develop  system  performance  (and  constraints)  specifications  and  enabling/critical  technologies  and 
verification  objectives. 

Develop  functional  definitions  for  enabling/critical  technologies  and  associated  verification  plan. 

Decompose  functional  definitions  into  critical  component  definition  and  technology  verification  plan. 

Develop  system  concepts,  including  enabling/critical  technologies.  Update  constraints  and  cost/risk 
drivers. 

Perform  Technology  Development  integration  and  verification/validation  activities. 

Demonstrate  enabling/critical  technology  components  versus  planned  technology  components. 
Demonstrate  system  functionality  versus  planed  functionality. 

Demonstrate/model  the  integrated  system  versus  the  performance  specification. 

Demonstrate  and  validate  the  system  concepts  and  technology  maturity  versus  defined  user  needs. 
Prepare  products/outputs  of  the  SE  processes  in  Technology  Development. 

Continue  conducting  AoA. 

Continue  development  of  TDS. 

Develop  Product  Support  Strategy. 

Provide  technical  support  to  development  of  CDD. 

Create  technical  exit  criteria. 

Conduct  technology  readiness  assessment. 

Conduct  independent  technology  assessment. 

Develop  C4I  support  plan. 

Conduct  system  threat  assessment. 

Provide  technical  input  to  refined  acquisition  strategy. 

Develop  TEMP. 

Prepare  Live-Fire  T&E  (LFT&E)  waiver  request  and  alternative  LFT&E  plan. 
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Compile  operational  test  agency  report  of  OT&E  results. 

Develop  program  protection  plan. 

Adhere  to  Clinger-Cohen  Act  Compliance  (MAIS  and  other  IT  systems  including  National  Security 
Systems). 

Prepare  Programmatic  ESOH  Evaluation. 

Identify  Low  Rate  Initial  Production  (LRIP)  quantities. 

Develop  independent  cost  estimate  and  manpower  estimate. 

Create  affordability  assessment. 

Conduct  component  cost  analysis. 

Create  CARD. 

Create  unit  cost  report. 

Reassess  EVM  system. 

Develop  Acquisition  Program  Baseline  (APB). 

Prepare  Selected  Acquisition  Report  (SAR). 

Adhere  to  spectrum  certification  compliance. 

Conduct  registration  of  mission-critical  and  mission-essential  information  systems. 

Conduct  technical  reviews  during  Technology  Development. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  SRR. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  Integrated  Baseline  Review  (IBR). 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  technology  readiness  assessment. 

System  Development  and  Demonstration 
Develop  the  functional,  allocated,  and  product  baselines. 

Develop  the  various  SE  related  inputs  and  outputs  of  the  SDD  phase. 

Conduct  tests  (development  test,  operational  test,  contractor,  live  fire,  acceptance,  etc.). 

Integrate  with  test  activities. 
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Assist  in  test  planning  and  design. 

Respond  to  issues  arising  during  test. 

Analyze  test  results. 

Evaluate  the  SE  product  and  processes  used  during  SDD. 

Perform  key  SE  activities  during  System  Integration. 

Interpret  user  capability  needs.  Refine  system  performance  specifications  and  environmental 
constraints. 

Develop  system  functional  specifications  and  system  verification  plan. 

Evolve  functional  performance  specifications  into  Configuration  Item  (Cl)  functional  (“design  to”) 
specifications  and  Cl  verification  plan. 

Evolve  Cl  functional  specifications  into  product  (“build  to”)  documentation  and  inspection  plan. 

Fabricate,  assemble,  code  to  “build  to”  documentation. 

Conduct  technical  reviews  during  System  Integration. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  IBR. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  SRR. 

Complete  definition  of  verification  requirements  should  be  an  integral  part  of  writing 
specifications  and  be  part  of  the  exit  criteria  for  the  formal  specification  review,  e.g.,  SRR  and 
software  specification  review. 

Require  all  test  requirements  be  identified  in  specifications  by  the  SRR. 

Employ  IMP  criteria  for  the  SRR  and  system  level  design  review  events  to  assure 
completeness. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  System  Functional  Review  (SFR). 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  PDR. 

Require  all  test  plans  be  in  place  to  satisfy  the  requirements  by  the  PDR. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  Critical  Design  Review 
(CDR). 

Require  a  full  comprehensive  integration  and  test  plan  as  entrance  criteria  for  CDR. 

Perform  key  SE  activities  during  System  Demonstration. 


K-30 


SYS  201 -Systems  Engineering  by  Acquisition  Phase 

Conduct  individual  Cl  verification  DT&E. 

Conduct  system  and  integration  DT&Es  and  Operational  Assessments  (OAs).  Verify  system 
functionality  and  constraints  compliance  to  specifications. 

Perform  combined  DT&E/OT&E.  Demonstrate  system  to  specified  user  needs  and  environmental 
constraints. 

Conduct  technical  reviews  during  System  Demonstration. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  Test  Readiness  Review  (TRR). 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  Flight  Readiness  Review. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  System  Verification  Review  (SVR). 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  Production  Readiness  Review 
(PRR). 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  technology  readiness  assessment. 
Prepare  products/outputs  of  the  SE  process  in  System  Demonstration. 

Update  documents  from  MS  B. 

Provide  technical  support  to  the  development  of  the  CPD. 

Production  and  Deployment 

Develop  the  various  SE  related  inputs  and  outputs  of  the  Production  and  Deployment  phase. 

Evaluate  manufacturing/production  planning  scheduling/control  systems. 

Understand  the  impact  of  production  rates  on  contractor  manning  processes. 

Evaluate  production  rate  capability. 

Conduct  learning  curve  evaluation. 

Analyze  impact  of  second  source/split  buy/competitive  production  contract. 

Determine  the  role  of  OT&E  in  the  acquisition  life  cycle. 

Use  SE  processes  to  monitor  and  control  the  system  configuration,  support  the  production  process,  and 
control  the  program  cost  and  schedule. 

Identify  the  ESOH  considerations  that  should  be  included  in  the  CPD. 

Perform  key  SE  activities  during  LRIP. 
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Analyze  deficiencies  to  determine  corrective  actions. 

Modify  configuration  (hardware,  software,  and  specifications)  as  necessary  to  correct  deficiencies. 
Verify  and  validate  production  configuration. 

Prepare  products/outputs  of  the  SE  process  in  LRIP. 

Conduct  Full  Rate  Production  Decision  Review. 

Update  documents  from  MS  C. 

Compile  beyond-LRIP  report. 

Develop  LFT&E  report. 

Prepare  component  LFT&E  report. 

Obtain  C4I  supportability  certification. 

Obtain  interoperability  certification. 

Conduct  post-deployment  performance  review. 

Conduct  technical  reviews  during  LRIP. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  IBR. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  Operational  Test  Readiness  Review 

(OTRR). 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  Physical  Configuration  Review 
(PCR). 

Conduct  technical  reviews  during  full  rate  production  and  deployment. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  IBR. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  PCR. 

Operations  and  Support  Phase 

Develop  the  various  SE  related  inputs  and  outputs  of  the  Operations  and  Support  phase. 

Monitor  experienced  system  performance  and  failures,  identify  root  causes,  evaluate  risks,  and  provide 
the  program  manager  with  an  integrated  technical  assessment  of  system  trends,  corrective  action 
alternatives,  staffing  and  resource  requirements. 

Understand  the  role  of  SE  in  product  improvements  during  development  and  the  post-production 
operations  and  support  phases. 
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Identify  any  problems  that  may  exist  in  a  product  improvement  scenario  and  how  the  SE  processes  are 
used  to  avoid  such  problems. 

Understand  that  product  improvement  is  implemented  within  the  SE  processes. 

Understand  product  improvements  for  legacy  systems  under  an  open  systems  approach. 

Identify  improvements  to  systems  for  the  purpose  of  Operations  and  Support  cost  reduction,  safety, 
replacing  obsolete  parts,  reliability,  tech  insertion,  etc. 

Use  SE  processes  to  implement  these  changes. 

Evaluate  use  of  the  SE  processes  to  reduce  risk  of  operational/support  problems. 

Understand  the  impact  of  design  on  the  operations  and  test  environment. 

Identify  sources  and  methodologies  for  technology  insertions. 

Understand  commercial  and  military  state  of  the  art  technology  applications. 

Know  and  understand  open  architecture  discipline,  tools,  methods  to  improve  aging 
systems/platforms  O&S  (specifically  for  tech  insertions). 

Know  methodologies  for  inserting  technology  upgrades  and  maintaining  technical  currency. 

Take  practical  courses  of  action  to  achieve  improved  performance,  cost  or  safety  in  weapon  systems  by 
modifying  existing  systems  and  taking  advantage  of  the  methodologies  which  permit  achieving  successful 
modification. 

Perform  key  SE  activities  during  Sustainment. 

Monitor  and  collect  all  service  use  data. 

Analyze  data  to  determine  root  cause  of  problem. 

Determine  the  risk/hazard  severity  of  the  system. 

Develop  corrective  action. 

Integrate  and  test  corrective  action. 

Assess  risk  of  improvements. 

Implement  and  field  systems. 

Prepare  products/outputs  of  the  SE  process  in  Sustainment. 

Create  modifications  to  fielded  systems. 

Provide  technical  support  for  the  updated  CDD  for  next  spiral  increment  (if  applicable). 
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Compile  data  for  Industrial  Security  Regulation  ( ISR). 

Conduct  technical  reviews  during  Sustainment. 

Prepare  for,  conduct  and  take  corrective  actions  resulting  from  in-service  review. 
Disposal 

Prepare  a  demilitarization  and  disposal  plan. 
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These  duties  and  tasks  are  divided  by  shaded  headings.  The  major  headings  are 
highlighted  in  dark  gray:  Total  Systems  View,  Organization,  Technical  Basis  for  Cost, 
Contract  Technical  Management,  Financial  Management,  International  Acquisition, 
Professional  Ethics  and  Leadership.  There  are  no  sub-headings  in  this  list. 

SYS  301--SE  Management  and  Leadership 

Evaluate  technical  issues,  assess  program  performance,  make  recommendations,  and  effectively  present 
these  issues  to  diverse  audiences. 

Build,  work  in,  motivate,  and  lead  high-performing  multidisciplinary  teams. 

Total  Systems  View 

Think  beyond  engineering  and  consider  all  functions  and  stakeholders  in  the  SE  processes. 

Understand  the  entire  acquisition  process. 

Organization 

Explain  the  barriers,  enablers,  organization  structure,  etc.  for  implementing  SE. 

Understand  the  roles  and  responsibilities  across  the  system  life  cycle. 

Maintain  a  robust  SE  organization. 

Respond  to  program  manager  and  functional  SE  leadership  regarding  the  proper  application  of  SE  to 
meet  program  objectives. 

Assign  a  lead  systems  engineer  at  program  inception  to  be  responsible  for  the  proper  application  of  SE  to 
meet  program  objectives. 

Staff  the  organic  SE  team  commensurate  with  the  scope  of  the  system,  sophistication/experience  of  the 
contractors,  and  integration  challenge  of  the  system. 

Operate  as  lead  engineer,  responding  directly  to  program  manager. 

Require  creation  of  a  single  technical  authority  that  is  responsible  for  the  overall  technical  effort  and 
empowered  to  resolve  technical  issues.  This  position  should  be  established  both  in  the  government 
program  office  and  the  contractor  program  office.  This  technical  authority  should  report  to  the  Program 
Director. 

Assess  and  improve  SE  processes. 

Technical  Basis  for  Cost 

Analyze  how  the  implementation  of  cost  containment  in  an  acquisition  program  supports  the  Cost  As  an 
Independent  Variable  (CAIV)  philosophy. 

Differentiate  the  effects  the  CAIV  strategy  has  on  the  cost  containment  efforts  (e.g.,  Total  Ownership  Cost 
(TOC),  Design-To-Cost  (DTC),  risk  management,  competition,  open  systems,  and  value  management. 

Recognize  the  composition  and  role  of  the  cost  performance-integrated  product  team  within  the 
acquisition  life  cycle. 
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Distinguish  cost  objectives  for  the  following:  Research,  Development,  Test  and  Evaluation  (RDT&E); 
procurement;  military  construction;  production;  operational  support:  disposal;  indirect  costs;  and 
infrastructure  tests. 

Recognize  how  contractor  incentives  influence  reaching  or  exceeding  cost  objectives. 

Select  metrics  to  track  and  monitor  progress  to  objectives. 

Implement  operational  cost  reduction. 

Know  and  understand  components  of  the  Reduction  of  Total  Ownership  Cost  (RTOC)  methodology. 
Assess  design  impact  on  total  operational  cost  and  identify  means  to  reduce  TOC. 

Understand  designing  for  change  using  techniques  such  as  open  systems  architectures. 

Understand  CAIV. 

Evaluate  the  system's  technical  basis  for  cost. 

Document  the  percentage  of  total  funding  to  be  spent  on  organic  and  outsourced  SE  efforts  in  system 
contracts. 

Examine  planned  programmed  resources  to  ascertain  if  adequate  resources  are  budgeted  to  execute  the 
proposed  SE  processes. 

Understand  why  and  when  technical  cost  estimates  are  needed. 

Identify  cost  structure  of  the  various  technical  cost  estimates. 

Perform  independent  technical  cost  estimates/analyses. 

Consider/identify  life  cycle  costs. 

Identify  technical  cost  estimating  methods. 

Be  able  to  do  parametric  analyses  with  respect  to  affordability  assessments/analyses. 

Apply  appropriate  technical  cost  estimation  method(s)  for  each  of  the  program  phases. 

Know  and  understand  affordability  assessment  techniques  and  tools. 

Develop  and  defend  program  engineering  requirements  and  cost  estimates  with  user  support. 

Understand  operating  and  support  cost  data  and  data  sources  and  their  differences. 

Understand  cost  estimation  tools/models  and  their  limitations. 

Contract  Technical  Management 

Understand  contractors'  SE  processes  and  perspectives. 

Require  contractors  to  address  how  the  SE  processes  will  be  applied  to  all  system  and  subsystem 
development  on  the  program  in  their  proposal,  including  how  they  will  be  applied  to  those  subsystems 
that  are  subcontracted.  These  SE  processes  must  be  described  in  the  proposal,  and  after  contract  award, 
published  in  a  program  document  available  to  all  program  participants  and  stakeholders.  Prime 
contractors  should  be  required  to  flow  down  a  requirement  for  defined,  written  SE  processes  to  their 
development  subcontractors. 
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Contracts  with  the  primes  should  be  structured  to  require  technical  engineering  management  visibility  and 
status  reporting  (metrics)  of  major  development  activities  at  the  subcontractor  level  and  should  be 
included  in  the  prime  contractor’s  reporting. 

Work  with  contractors  and  provide  informed  assessments  of  their  progress. 

Certify  technical  progress. 

Develop  contract  documents. 

Document  the  requirements  for  the  SE  processes  by  phase  in  system  contracts. 

Distinguish  responsibilities  of  contracting  officer  vs.  engineer,  including  contracting  officer's  technical 
representative. 

Capitalize  on  opportunities  to  develop  performance  based  solicitations  for  systems. 

Develop  performance  expectations,  incentives  and  metrics  to  describe  acquisition  needs  and  evaluate 
outcomes. 

Financial  Management 

Understand  the  Planning,  Programming,  Budgeting,  and  Execution  System  (PPBES)  sources  and  uses  of 
funds,  and  how  budget  issues  impact  the  program. 

Manage  budget  effects  on  the  technical  program. 

International  Acquisition 

Understand  International  Acquisition  (IA)  policy  and  techniques. 

Utilize  offshore  technology  in  system  design  where  it  provides  a  benefit. 

Assess  security  impacts  of  international  acquisition. 

Assess  ESOH  impacts  of  international  acquisitions. 

Assess  releasability  of  candidate  technologies. 

Ensure  standardization,  interoperability. 

Identify  the  benefits  and  pitfalls  in  international  acquisition  from  a  SPRDE  manager's  perspective. 
Professional  Ethics 
Employ  professional  ethics. 

Differentiate  judgments,  legal  issues,  and  ethical  issues. 

Evaluate  ethical  issues  in  engineering  and  business  practices. 

Leadership 

Plan  and  know  your  leadership  objectives. 

Provide  resources  for  workers  (e.g.,  money,  people,  and  facilities. 

Reward  and  discipline  appropriately. 
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Act  as  "mentor"  for  leaders  of  tomorrow. 

Foster  technical  training  and  education. 

Develop  a  plan  for  professional  SE  training  and  development. 

Train  the  program  participants  in  the  execution  of  SE  processes,  including  the  value  of  the  processes  and 
the  importance  of  applying  the  SE  processes. 

Establish  a  program  and  process  for  incentivizing  career  SE  positions  within  the  government. 

Promote  technical  career  development  and  growth. 
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SPRDE/SE  Performance  Objectives  (POs) 


SPRDE/SE  PERFORMANCE  OBJECTIVES 


SPRDE/SE  Performance  Objectives 

Key:  Dark  gray  high  level  topic;  Light  gray  subtopics 

Total  Systems  View- 

Articulate  the  value  of  systems  engineering  in  the  acquisition  process 
Acquisition 

Apply  the  appropriate  Defense  acquisition  policies  and  SE  processes  necessary  for  successful 
execution  of  phase  activities 

Evolutionary  Acquisition  (EA) 

Match  evolutionary  requirements  with  capability  needs 

IPPD 

Apply  the  IPPD  principles  within  the  SE  processes  as  performed  by  Integrated  Product  Teams 
(IPTs) 

Leadership 

Evaluate  leadership  of  contractor  and  government  technical/management  IPTs 
Organization 

Explain  how  to  implement  and  sustain  a  robust  systems  engineering  organization. 

Financial  Management 

Understand  the  Planning,  Programming,  Budgeting,  and  Execution  System  (PPBES)  sources 
and  uses  of  funds,  and  how  budget  issues  impact  the  program. 

Contract  Technical  Management 

Evaluate  contractors'  SE  processes  and  perspectives  in  proposals 

Technical  Basis  for  Cost 

Evaluate  the  system's  technical  basis  for  cost. 

International  Acquisition  (IA) 
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Identify  the  benefits  and  pitfalls  in  international  acquisition  from  a  SPRDE  manager's 
perspective 

Professional  Ethics 

Evaluate  ethical  issues  in  engineering  and  business  practices 
SE  Processes 

Apply  the  systems  engineering  processes  to  transform  requirements  and  constraints  into  an 
operational  system  design 

Requirements  Development 

Develop  a  set  of  functional,  physical,  and  operational  requirement  viewpoints  in  accordance 
with  SE  standards. 

Requirements  Management 

Implement  a  disciplined  requirements  management  process  from  the  JCIDS  outputs. 
Architectural  Design 

Understand  the  evolution  of  the  architecture  from  the  established  system  requirements. 

Design  Solution 

Translate  outputs  of  the  Architectural  Design  process  into  system  product  and  process  designs 
which  duly  consider  life  cycle  issues. 

Implementation 

Understand  the  purpose,  inputs  and  outputs  of  the  Implementation  process 
Integration 

Understand  the  purpose,  inputs  and  outputs  of  the  Integration  process 

Reinforce  the  importance  of  specifying,  allocating,  and  controlling  interface  requirements 
including  interfaces  among  the  members  of  a  SoS/FoS. 

Verification 

Apply  the  Verification  process  in  order  to  ensure  the  system  elements  meet  or  exceed  their 
requirements. 
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Validation 

Understand  the  purpose,  inputs  and  outputs  of  the  Validation  process 
Transition 

Understand  the  purpose,  inputs  and  outputs  of  the  Transition  process 
Technical  Planning 

Differentiate  the  role  of  technical  planning  within  the  systems  engineering  effort  from  overall 
program  planning 

Technical  Assessment 

Assess  programs  and  identify  risks  and  pitfalls 
Decision  Analysis 

Apply  quantitative  and  qualitative  tools  to  support  problem  solving  and  decision  making  in  an 
acquisition  environment 

Risk  Management 

Apply  the  risk  management  process  within  an  IPPD  environment  in  order  to  determine  program 
technical  risk. 

Configuration  Management 

Understand  the  configuration  baselines  (functional,  allocated,  and  product ),  when  they  are 
established,  and  why  they  are  important. 

Data  Management 

Contrast  the  roles  and  relationships  of  configuration  management  (CM),  data  management, 
and  interface  management. 

Interface  Management 

Differentiate  the  following  interface  management  function  elements:  Interface  control  working 
groups  (ICWG)  and  interface  control  documentation  (ICD). 

SE  by  Phases 

Understand  the  systems  acquisition  life  cycle  phases  and  the  major  SE  activities  to  be 
accomplished  in  each  phase 
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Solicitation  and  Source  Selection 

Explain  how  the  SOW  and  RFP  are  evaluated  from  an  SE  perspective 
Concept  Refinement 

Develop  the  various  SE  related  inputs  and  outputs  of  the  Concept  Refinement  phase 
Technology  Development 

Develop  the  various  SE  related  inputs  and  outputs  of  the  Technology  Development  phase 
System  Development  and  Demonstration 

Develop  the  various  SE  related  inputs  and  outputs  of  the  SDD  phase 
Production  and  Deployment 

Develop  the  various  SE  related  inputs  and  outputs  of  the  Production  and  Deployment  phase 

Use  the  SE  processes  to  monitor  and  control  the  system  configuration,  support  the  production 
process,  and  control  the  program  cost  and  schedule 

Operations  and  Support  Phase 

Develop  the  various  SE  related  inputs  and  outputs  of  the  Operations  and  Support  phase 
Evaluate  use  of  the  SE  processes  to  reduce  risk  of  operational/support  problems 
Disposal 

SE  Techniques  and  Tools 

Understand  and  be  able  to  use  SE  tools  and  methods 
SEP 

Develop/coordinate  and  implement  the  SEP 
WBS 

Recognize/Understand  the  Work  Breakdown  Structure  (WBS)  and  its  application  throughout 
the  systems  engineering  processes 

Value  Engineering 
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Understand  value  engineering  methods  and  tools 
Technical  Performance  Measurement 

Determine  the  role  of  technical  performance  measurements  (TPMs)  in  the  systems  engineering 
processes 

Trade  Studies 

Understand  the  key  alternatives  or  tradeoffs  made  as  the  system  evolves  through  its  life  cycle 
from  the  conceptual  level  of  development  to  the  detailed  subsystem  level  of  development. 

Modeling  and  Simulation 

Identify  and  assess  the  M&S  requirements,  benefits,  pitfalls,  planning  and  applications  in 
systems  acquisition 

Prototyping 

Understand  the  role  of  prototyping  in  spiral  development 
Robust  Engineering 

Understand  robust  design  techniques  and  tools 
Modular  Open  Systems  Architecture 

Understand  the  Modular  Open  Systems  Architecture  technique 

TEMP 

Understand  the  Test  &  Evaluation  Master  Plan  (TEMP),  and  other  related  documents  and  their 
relationship  to  the  SEP  and  the  acquisition  process. 

Technical  Data  Packages 

Understand  the  purpose  and  timing  of  technical  data  packages  and  other  system  specific 
information  over  the  lifecycle 

Specifications 

Understand  the  role  of  specifications  over  the  lifecycle 
Earned  Value  Management 

Understand  Earned  Value  (EV)  principles,  evaluate  EV  data,  and  make  recommendations 
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Technical  Reviews 

Associate  the  types  of  major  reviews  with  the  issues  to  be  addressed  in  each,  and  each 
review’s  relationship  to  the  acquisition  life  cycle. 

Design  Considerations 

Open  Systems  Architecture 

Understand  open  systems  architectures,  disciplines,  tools  and  methods  and  application  to 
system  interoperability 

Architectures  and  Interoperability 

Determine  the  purpose  and  timing  of  architectures  over  the  life  cycle 
Software 

Recognize  the  complexity  of  software  development,  its  integral  nature  to  the  Systems 
Engineering  processes,  and  top-level  "best  practices"  for  successful  software  development 

COTS  and  NDI 

Conduct  SE  analyses  of  the  reuse  and  COTS  candidates  to  determine  that  the  candidates 
exist  in  acceptable  usable  form,  that  they  meet  the  new  system  requirements,  and  that  they  are 
well  documented  and  designed  for  reuse. 

Manufacturing  Capability  and  Producibility 

Recognize  the  major  producibility  goals  of  the  design  effort  and  the  DoD  quality  process  which 
translates  a  released  design  to  a  producible  product 

Determine  the  role  of  manufacturing  capability  in  the  systems  engineering  process  throughout 
the  acquisition  life  cycle 

Quality  Assurance 

Awareness  of  system  quality  methods? 

Reliability,  Availability  and  Maintainability 

Identify  the  impact  of  reliability,  availability,  maintainability  on  system  support  and  total 
ownership  costs 

Human  Systems  Integration 
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Understand  human  and  cognitive  factors  and  their  affect  on  the  system  technical  effort 
ESOH 

Understand  ESOH  Statutes,  Regulations,  Policies,  International  Agreements,  and 
Government/Industry  Standards  and  how  they  affect  Systems  Acquisition  Programs  through 
the  Life-cycle 

Understand  how  to  Integrate  ESOH  Considerations  into  the  SE  processes 
Survivability 

Develop  survivability  requirements 
Corrosion  Prevention  and  Control 

Understand  how  corrosion  prevention  and  control  planning  impacts  system  requirements 
Disposal  and  Demilitarization 

Understand  the  goals  and  objectives  of  disposal  and  demilitarization  and  their  impact  on  the 
SE  processes  during  the  development  of  a  weapon  system 

Information  Assurance 

Incorporate  information  assurance  requirements  into  system  design  activities  to  ensure 
availability,  integrity,  confidentiality,  authentication  and  non-repudiation  of  critical  system 
information 

System  Security 

Understand  how  system  security  engineering  impacts  system  requirements 
Test  and  Evaluation 

Determine  the  role  of  test  and  evaluation  (T&E)  in  the  SE  and  acquisition  management 
processes 

Sustainment 

Include  sustainment  as  a  topic  throughout  the  design  process;  include  sustainment  specialists 
in  program  planning  activities. 

Determine  how  acquisition  logistics  activities  impact  and  relate  with  SE  and  other  functional 
areas  within  the  acquisition  life  cycle 
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LEARNING  OUTCOMES 


SPRDE/SE  CAREER  PATH  LEARNING  OUTCOMES  BY  CATEGORIES  AND  TOPICS 

LO# 

Level  1 

Level  II 

Level  III 

TOTAL  SYSTEMS  VIEW 

Acquisition 

1 

Recognize  SE  best  practice  and 
processes. 

Evaluate  SE  best  practice  and  processes. 

Compare  SE  best  practice  and  processes. 

2 

Structure  SE  processes  to  phase  objectives 
and  phase  products  (including  in-service 
systems  engineering). 

Structure  SE  processes  to  phase 
objectives  and  phase  products  (including 
in-service  systems  engineering). 

3 

Distinguish  the  relationship  of  program 
technical  performance  objectives  to  program 
budgets  to  program  schedule  as  well  as  the 
methods  required  to  assess  the  technical 
risks  driving  cost/schedule/performance  risk. 

Evaluate  the  relationship  of  program 
technical  performance  objectives  to 
program  budgets  to  program  schedule  as 
well  as  the  methods  required  to  assess  the 
technical  risks  driving 
cost/schedule/performance  risk. 

4 

Determine  the  universe  of  performance 
requirements  and  constraints  (technical  and 
programmatic)  that  must  be  addressed  and 
managed  in  the  phase. 

Derive  the  universe  of  performance 
requirements  and  constraints  (technical 
and  programmatic)  that  must  be 
addressed  and  managed  in  the  phase. 

Evolutionary  Acquisition  (EA) 

5 

Analyze  how  evolutionary  acquisition  (e.g. 
incremental  development)  can  be 
implemented  in  SE  to  achieve  program 
objectives. 

Assess  how  evolutionary  acquisition  (e.g. 
incremental  development)  can  be 
implemented  in  SE  to  achieve  program 
objectives. 
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LO# 

Level  1 

Level  II 

Level  III 

TOTAL  SYSTEMS  VIEW 

6 

Determine  how  incremental 
development/spiral  development  affects  the 
technical  aspects  of  the  acquisition  strategy 
as  well  as  how  they  affect  the  technical 
nature  of  the  SE  phase  products. 

Evaluate  how  incremental 
development/spiral  development  affects 
the  technical  aspects  of  the  acquisition 
strategy  as  well  as  how  they  affect  the 
technical  nature  of  the  SE  phase  products. 

7 

Examine  technical  risk  as  a  major 
determinant  of  planned  evolutionary 
approaches. 

Evaluate  technical  risk  as  a  major 
determinant  of  planned  evolutionary 
approaches. 

8 

Relate  the  quality  of  the  phase's  technical 
products  to  the  risk  mitigation  of  follow-on 
phases. 

Recommend  the  quality  of  the  phase's 
technical  products  to  the  risk  mitigation  of 
follow-on  phases. 

9 

Determine  the  appropriate  systems 
engineering  efforts  required  to  achieve 
acceptable  levels  of  risk  for  entry  into  the 
next  acquisition  phase. 

Critique  the  appropriate  systems 
engineering  efforts  required  to  achieve 
acceptable  levels  of  risk  for  entry  into  the 
next  acquisition  phase. 

10 

Identify  the  systems  engineering  efforts 
required  in  each  acquisition  phase. 

Contrast  the  systems  engineering  efforts 
required  of  the  acquisition  phase  to  the 
acquisition  products  required  of  that  phase. 

Formulate  the  relationship  of  the  systems 
engineering  efforts  required  of  the 
acquisition  phase  to  the  acquisition 
products  required  of  that  phase. 

11 

Determine,  based  on  program  technical 
scope  and  risk,  the  level  of  systems 
engineering  required  and  the  subject  matter 
expertise  required  to  execute. 

Evaluate,  based  on  program  technical 
scope  and  risk,  the  level  of  systems 
engineering  required  and  the  subject 
matter  expertise  required  to  execute. 

IPPD 

12 

Apply  IPPD  principles  to  the  conduct  of  SE 
as  applied  to  an  acquisition  program. 

Assess  proper  implementation  of  IPPD 
principles  in  conducting  SE  during  an 
acquisition  program. 
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13 

Examine  the  implementation  of  IPPD  on 
government  IPTs  and  integrate  the  approach 
to  the  contractor's  implementation  of  IPPD. 

Recommend  the  implementation  of  IPPD 
on  government  IPTs  and  integrate  the 
approach  to  the  contractor's 
implementation  of  IPPD. 

14 

Determine  processes  and  products  linkage 
on  IPTs. 

Assess  processes  and  products  linkage  on 

IPTs. 

15 

Structure  program  SE  implementation  to 
provide  for  cross-IPT  controls  of 
requirements  and  constraints  budgets. 

Assess  program  SE  implementation  for 
proper  cross-IPT  controls  of  requirements 
and  constraints  budgets. 

16 

Recognize  the  subject  matter  expertise 
required  on  IPTs  to  implement  IPPD  across 
the  full  spectrum  of  requirements  and  design 
considerations. 

Recommend  the  subject  matter  expertise 
required  on  IPTs  to  implement  IPPD 
across  the  full  spectrum  of  requirements 
and  design  considerations. 

17 

Justify  IPPD  tailored  to  acquisition  phase, 
providing  for  required  integration  across 
the  system,  across  IPTs,  and  across 
subject  matter  domains. 

Leadership 

18 

Arrange  proper  control  and  reporting 
mechanisms  for  leadership  of  the  overall 
technical  effort,  for  systems  engineering, 
for  requirements  management,  and  for 
systems  integration. 

19 

Evaluate  process  and  product  oversight 
mechanisms  to  ensure  insight  to  technical 
risk  at  each  technical  phase  of  effort. 
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20 

Assess  implementation  of  technical  review 
chairmanship/leadership  to  include  the 
application  of  technical  review  best 
practice  (entry/exit  criteria,  evaluation  of 
program/product  technical  maturity, 
determination  of  critical  participants,  and 
conduct  of  technical  risk  assessment 
against  objective  measures). 

Organization 

21 

Determine  how  IPTs  should  be  structured  to 
meet  technical  objectives. 

Determine  how  IPTs  should  be  structured 
to  meet  technical  objectives. 

22 

Determine  how  IPTs  should  be  staffed  to 
address  technical  scope  and  risk,  systems 
development,  systems  integration,  and  full 
application  of  systems  engineering. 

23 

Determine  IPT  staffing  required  to  address 
allocated  requirements  and  design 
constraints  (budgets)  using  cross¬ 
functional  teams. 

24 

Apply  cross-IPT  mechanisms  for  functional 
and  system  integration. 

Construct  cross-IPT  mechanisms  for 
functional  and  system  integration. 

25 

Identify  mechanisms  for  cross-IPT 
management  of  KPPs,  critical  operation 
requirements,  critical  budgets  (weight, 
power,  etc.). 

Construct  mechanisms  for  cross-IPT 
management  of  KPPs,  critical  operation 
requirements,  critical  budgets  (weight, 
power,  etc.). 
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Financial  Management 

26 

Recognize  the  technical  aspects  of  a 
program  and  how  these  relate  to  the  cost 
accounts  set  up  in  the  Earned  Value  system. 

Consider  the  technical  aspects  of  a 
program  and  how  these  relate  to  the  cost 
accounts  set  up  in  the  Earned  Value 
system. 

27 

Recognize  what  types  of  technical  effort 
need  to  be  tracked  as  earned  value  to  make 
the  EVM  system  effective  and  importance  of 
minimizing  use  of  level-of-effort  cost 
accounts. 

Recognize  what  types  of  technical  effort 
need  to  be  tracked  as  earned  value  to 
make  the  EVM  system  effective  and 
importance  of  minimizing  use  of  level-of- 
effort  cost  accounts. 

28 

Use  EVM  data  to  track  technical 
performance  and  use  as  a  decision¬ 
making  mechanism  for  resource  re¬ 
allocation. 

29 

Assess  technical  adequacy  of  cost 
accounts  versus  technical  risk,  technical 
scope,  and  technical  uncertainty. 
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Level  III 
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Contract  Technical  Management 

30 

Recognize  offerors’  descriptions  of  key 

SE  processes  and  practices  as 
presented  in  proposals 

Evaluate  the  offeror's  proposed  approach 
in  the  area  of  SE  to  include:  1 )  the  level  of 
capability  maturity  in  SE  and  other 
technical  process  areas,  2)  the 
understanding  of  SE  processes  and 
standards  to  be  used,  3)  appropriateness 
of  SE  process  application  to  the  phase,  4) 
adequacy  of  technical  baseline  products 
as  development  control  mechanisms,  5) 
technical  reviews  as  integrated  (program 
team  and  independent  subject  matter 
expert)  technical  assessment  tied  to 
objective  entry/exit  criteria,  6)  independent 
technical  chair  for  reviews,  7)  subject 
matter  expertise  appropriate  for  the  scope 
and  risk  of  the  program. 

31 

Determine  contractual  content  required  for 
proper  application  of  SE  to  include 
incentives  for  SE  performance  (ensure 
that  progress  payments  are  not  tied  to 
technical  review  completion)  and 
provisions  for  SE  product  and  technical 
information  availability. 

32 

Identify  pitfalls  of  tying  progress  payments  to 
technical  reviews  completion. 

33 

Understand  the  various  types  of 
contractor  financing  to  include  progress 
payments 

Ensure  that  progress  payments  are  not  tied 
to  technical  review  completion.  Determine 
the  best  methods  to  finance  contractor 
efforts 

Assess  the  effectiveness  of  the  various 
types  of  contractor  financing  options 
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34 

Develop  an  SE  surveillance  plan 

Assess  contractor’s  SE  performance 

35 

Define  contractor  system  processes 

Determine  contractor’s  SE  system  processes 

Predict  contractor’s  SE  performance 

36 

Recognize  contractor  SE  standards  and 
processes  that  are  applied  in  each 
acquisition  phase 

Develop  an  SE  surveillance  plan  based  on 
those  standards 

Evaluate  contractor’s  performance  based 
on  the  phase  of  the  program. 

37 

Describe  the  technical  baseline  products 

Examine  the  relationships  between  the 
technical  baseline  products  and  the  SE 
processes. 

Evaluate  contractor’s  performance  based 
on  the  baseline  products  and  SE 
processes 

38 

Define  entry/exit  criteria 

Determine  measures  for  entry/exit  criteria 

Integrate  the  entry/exit  criteria  measures 
into  the  SE  performance  plan 

39 

Understand  purpose  and  content  of 
technical  reviews 

Determine  the  different  types  of  technical 
reviews  during  each  phase  of  the  acquisition 
process  and  the  output  of  these  reviews 

Assess  the  contactor’s  performance  based 
on  the  outcome  of  these  technical  reviews 

40 

Understand  how  SE  staffing  level 
change  during  each  acquisition  phase. 
Identify  the  tasks  to  be  completed  by  the 
System  engineer 

Determine  the  appropriateness  of  the 
contractor’s  SE  staffing  levels. 

Integrate  the  resourcing  risk  in  to  the  SE 
performance  plan 

41 

Identify  the  different  types  of  contracts 
awarded  to  incentivize  SE 

Determine  the  appropriate  level  of  SE 
incentive 

Assess  the  effectiveness  of  putting  SE 
incentives  in  contracts 
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42 

Determine  the  types  of  technical  information 
that  should  be  contractually  available  for 
review 

Evaluate  contractor’s  performance  based 
on  technical  information. 

43 

Define  SE  metrics 

Determine  appropriate  SE  metrics 

Predict  contractor  performance  based  on 
product  and  process  metrics 

44 

Explain  how  the  SOW  and  RFP  are 
evaluated  from  an  SE  perspective 

Technical  Basis  for  Cost 

45 

Identify  cost  drivers  that  affect  the 
development  of  cost  estimates  and  program 
budgets. 

Apply  knowledge  of  cost  drivers  to  affect 
the  development  of  cost  estimates  and 
program  budgets. 

46 

Consider  the  effects  of  cost  driver 
uncertainty  on  budget  uncertainty  and 
program  risk. 

47 

Apply  systems  engineering  best  practice 
to  define  technical  cost  drivers  on  the 
program,  commensurate  with  program 
phase  requirements. 

48 

Construct  the  linkage  of  cost  drivers  to 
other  aspects  of  the  program  (cost, 
schedule,  performance)  and  articulate  the 
relationships. 
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International  Acquisition  (IA) 

49  Identify  the  benefits  and  pitfalls  in  Assess  the  benefits  and  pitfalls  in 

international  acquisition  from  a  SPRDE  international  acquisition  from  a  SPRDE 

manager's  perspective.  manager's  perspective. 

Professional  Ethics 

50  Identify  ethical  issues  in  engineering  and  Assess  ethical  issues  in  engineering  and  Evaluate  ethical  issues  in  engineering  and 

business  practices  business  practices  business  practices. 
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SE  PROCESSES 

General 

51 

Recognize  the  national  and  international 
SE  process  standards  in  the  conduct  of 

SE  in  DoD  acquisition 

Distinguish  SE  process  models  in  national 
and  international  SE  process  standards 
from  life  cycle  models  (waterfall,  Vee,  spiral) 

Assess  the  national  and  international 

SE  process  standards  in  the  conduct  of  SE 
in  DoD  acquisition 

52 

Revise  and  tailor  the  SE  processes  in  the 
national  and  international  SE  process 
standards  to  an  individual  program 

TECHNICAL  MANAGEMENT  PROCESSES 

Decision  Analyses 

53 

Describe  the  purpose,  inputs  and  outputs 
of  the  Decision  Analysis  process 

Apply  the  Decision  Analysis  process  during 
each  phase  of  an  acquisition  program 

Evaluate  group  decision  making  and  voting 
procedures  in  an  IPPD  environment  on  an 
acquisition  program 

54 

Apply  quantitative  and  qualitative  tools  to 
support  problem  solving  and  decision 
making  in  an  acquisition  environment 

55 

Prepare  documentation  of  decision  rationale 

Technical  Planning 

56 

Describe  the  purpose,  inputs  and  outputs 
of  the  Technical  Planning  process 

Apply  the  Technical  Planning  process 
during  each  phase  of  an  acquisition 
program 

Assess  contractor  technical  planning 
and/or  execution  efforts  on  an  acquisition 
program 
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57 

Explain  the  benefits  of  event  criteria 
planning  over  calendar  time-based 
scheduling 

Apply  control  criteria  and  metrics  to  the 
Technical  Planning  process 

58 

Recognize  the  role  of  technical  planning 
with  the  SE  effort  from  overall  program 
planning 

Categorize  the  sensitivity  of  interrelated 
tasks  in  the  Systems  Engineering  Plan 

Technical  Assessment 

59 

Describe  the  purpose,  inputs,  and 
outputs  of  the  Technical  Assessment 
process 

Apply  the  Technical  Assessment  process 
during  all  phases  of  an  acquisition  program 

Assess  an  acquisition  program  to  identify 
risks  and  pitfalls 

60 

Use  Measures  of  Performance  and 
Effectiveness  for  proposed  materiel 
solutions 

Evaluate  the  contractor’s  technical 
management  of  their  systems  and 
processes  (e.g.  engineering, 
manufacturing,  and  quality)  on  an 
acquisition  program 

Requirements  Management 

61 

Describe  the  purpose,  inputs,  and 
outputs  of  the  Requirements 

Management  process 

Apply  the  Requirements  Management 
process  during  all  phases  of  an  acquisition 
program,  e.g.,  from  the  ICD  to  the  RFP 
performance  requirements  document,  to 
contract  specifications,  to  product 
specifications,  etc. 

Create  plans  to  ensure  that  functional, 
design,  performance,  environment,  cost 
and  schedule  requirements  are  being 
tracked 

62 

Recognize  the  effective  application  of 
commercially  available  software  tools  to 
implement  the  requirements  management 
and  tracking  process 

For  each  baseline  specification,  develop 
program  metrics  to  measure  and  track  the 
extent  of  requirements  changes  in  terms  of 
number  of  requirements  “shall  statements” 
changed  from  contract  award  forward 
through  the  development  cycle 
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63 

Relate  lower  level  specifications  to  higher 
level  requirements 

Assess  a  contractor’s  implementation  of 
requirements  tracking  tools  and  their 
capability  in  flowing  down  these  tools  to 
the  subcontractor  base  to  assure 
completeness  on  an  acquisition  program 

Risk  Management 

64 

Describe  the  purpose,  inputs  and  outputs 
of  the  Risk  Management  process 

Apply  the  Risk  Management  process  to  an 
acquisition  program  within  an  IPPD 
environment  in  order  to  determine  program 
technical  risk  and  as  a  basis  for  making 
sound  acquisition  program  decisions 

Assess  interrelationship  of  technical  risk 
with  cost  and  schedule 

65 

Describe  how  risk  management  is  used 
to  reduce  operational  and  support 
problems 

Apply  risk  management  methodologies  and 
tools 

Develop  risk  management  plans  for 
prototyping,  spiral  development, 
evolutionary  acquisition,  modifications  and 
upgrades 

66 

Summarize  risk  mitigation  techniques 
(NDI,  COTS,  TAA,  TAP) 

Demonstrate  early  simulation 
demonstrations  of  reuse  candidates  in  the 
risk  management  program  as  risk  mitigation 
activities 

Evaluate  alternative  approaches  for 
moderate/high  risks 

67 

Explain  risk  reporting  system  and 
tracking  techniques 

Configuration  Management 

68 

Describe  the  purpose,  inputs,  and 
outputs  of  the  Configuration 

Management  process 

Apply  the  Configuration  Management 
process  during  all  phases  of  an  acquisition 
program 

Create  the  following  configuration 
identification  function  elements: 
configuration  items;  configuration 
documentation;  configuration  baselines; 
and  program-unique  specifications 
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69 

Explain  the  configuration  baselines 
(functional,  allocated,  and  product)  by 
when  they  are  established  and  why  they 
are  important 

Apply  the  following  configuration  control 
function  elements:  engineering  change 
proposals  (ECP);  request  for  deviation 
(RFD);  and  configuration  control  boards 

Plan  baseline  management  including 
audits  for  a  system  in  an  acquisition 
program 

70 

Summarize  the  activities  of  a 

Configuration  Control  Board  (CCB) 

Analyze  a  change  process  in  an  acquisition 
program  and  track  the  engineering  changes 

Generate  Configuration  Baselines 
(functional,  allocated,  and  product)  at  the 
appropriate  time  in  the  program 

71 

Summarize  the  technical  support  for 
developing  Acquisition  Program 

Baselines 

Technical  Data  Management 

72 

Describe  the  purpose,  inputs,  and 
outputs  of  the  Technical  Data 

Management  process 

Apply  the  Technical  Data  Management 
process  during  all  phases  of  an  acquisition 
program 

Evaluate  the  quality  of  the  technical  data 
on  an  acquisition  program 

73 

Define  minimum  essential  life  cycle 
technical  data  requirements 

Trace  technical  data  to  requirements 

Prepare  technical  data  base  in  an 
integrated  data  environment 

Interface  Management 

74 

Describe  the  purpose,  inputs,  and 
outputs  of  the  Interface  Management 
process 

Apply  the  Interface  Management  process 
during  all  phases  of  an  acquisition  program 

Evaluate  the  integrity  of  interface  controls 
on  an  acquisition  program 

75 

Explain  the  interface  management 
process  functions  of  the  interface  control 
working  group  (ICWG)  and  interface 
control  documentation 

Compare  the  roles  of  configuration 
management,  technical  data  management, 
and  interface  management 

Evaluate  the  treatment  of  the  user's 
interoperability  requirements  for  a  system 
in  an  acquisition  program 

76 

Examine  the  relationships  among 
configuration  management,  technical  data 
management,  and  interface  management 

Create  interface  control  documentation  for 
a  system  in  an  acquisition  program 
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TECHNICAL  PROCESSES 

Requirements  Development 

77 

Describe  the  purpose,  inputs,  and 
outputs  of  the  Requirements 

Development  Process 

Apply  the  Requirements  Development 
process  during  all  phases  of  an  acquisition 
program 

Develop  requirements  in  an  integrated 
framework 

78 

Identify  all  lifecycle  stakeholders  and  the 
types  of  requirements  from  each 

Analyze  requirements  flow  down  and 
allocation  into  specifications  for  a  system  in 
an  acquisition  program 

Create  derived  system  requirements 
expressed  in  terms  of  operational, 
functional,  and  physical  viewpoints  from 
the  performance  parameters  in  capability 
documents  (ICD,  CCD,  CPD) 

79 

Describe  lifecycle  impacts  and  their 
relationship  to  requirements  for  a  system 
in  an  acquisition  program 

Analyze  behavioral  requirements  and 
design  constraints  for  a  system  in  an 
acquisition  program 

Logical  Analysis 

80 

Describe  the  purpose,  inputs  and  outputs 
of  the  Logical  Solution  process 

Apply  the  Logical  Solution  process  during  all 
phases  of  an  acquisition  environment 

Evaluate  the  preferred  architecture  for  a 
system 

81 

Identify  solution  alternatives  (training, 
doctrine,  material,  etc.) 

Apply  tools  used  in  the  Logical  Solution 
process  (functional  flow  block  diagram, 
timeline  sheets,  and  requirements  allocation 
sheet) 

Create  a  behavior  model 

82 

Recognize  functional  analysis,  functional 
allocation,  and  functional  architecture 
activities 

Demonstrate  traceability  from  logical 
groupings  (of  functions,  etc.)  to 
requirements 

Develop  a  functional  architecture  and  the 
functional  specifications  for  a  system  in  an 
acquisition  program 
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Design  Solution 

83 

Describe  the  purpose,  inputs  and  outputs 
of  the  Design  Solution  process 

Apply  the  Design  Solution  process  during  all 
phases  of  an  acquisition  program 

Generate  the  physical  architecture  for  a 
system  and  allocate  functions  to  physical 
elements 

84 

Describe  how  outputs  of  the  Logical 
Solution  process  are  translated  into 
system  product  and  process  designs  that 
duly  consider  life  cycle  issues 

Analyze  alternative  design  solutions  and 
select  one 

85 

Recognize  the  proper  completion  of  the 
top  level  system  design  and 
requirements  allocation  to  subsystems 
prior  to  initiation  of  the  subsystem  design 
activity 

Apply  the  tools  and  techniques  of  the 

Design  Solution  process  (e.g.,  concept 
description  sheets  and  schematic  block 
diagrams) 

Develop  specification  trees  and 
specifications  for  a  system  in  an 
acquisition  program 

86 

Identify  the  role  of  industry  standards  in 
describing  the  physical  architecture  of  a 
system 

87 

Analyze  outputs  of  modeling  and  simulation 
(M&S)  activities  during  the  Design  Solution 
process 

88 

Develop  the  M&S  strategy  and  detailed 
approach  for  a  system  in  an  acquisition 
program. 

Implementation 

89 

Describe  the  purpose,  inputs,  and 
outputs  of  the  Implementation  process 

Apply  the  Implementation  process  during 
each  phase  of  an  acquisition  program 

Develop  documentation  for  system 
elements  in  an  acquisition  program 
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90 

Define  the  major  variables  and  trends 
encountered  in  production  and  how  they 
relate  to  other  functional  areas 

Analyze  the  make/buy/reuse  decisions  for  a 
system  in  an  acquisition  program 

Develop  the  manufacturing  strategy  and 
manufacturing  plan  for  a  system  in  an 
acquisition  program 

91 

Explain  the  activities  in  process 
inspection  and  product  acceptance 

Analyze  a  manufacturing  system 
architecture 

92 

Summarize  acceptance  testing  activities 

Integration 

93 

Describe  the  purpose,  inputs,  and 
outputs  of  the  Integration  process 

Apply  the  Integration  process  during  each 
phase  of  an  acquisition  program 

Evaluate  impact  of  new  system  to  current 
fielded  solutions 

94 

Recognize  system  interfaces 

(S  of tware/Software, Hardware/S  oft  ware, 

Human) 

Demonstrate  the  compatibility  (or  lack  of)  all 
functional  and  physical  interfaces  for  a 
system  in  an  acquisition  program 

Develop  a  system  hardware  and  software 
integration  plan  for  a  system  in  an 
acquisition  program 

95 

Describe  system/subsystem  interface 
requirements  for  a  system  in  an 
acquisition  program 

Analyze  the  integration,  testing,  and 
modification  of  the  physical  product  from 
lowest  level  through  system  level  (test, 
analyze  and  fix) 

Manage  the  definition/refinement  of 
interfaces  as  the  design  matures  for  a 
system  in  an  acquisition  program 

96 

Explain  the  benefits  of  conducting 
system-level  integration  and  test 
planning  early  in  the  program 

Evaluate  the  specification,  allocation,  and 
controlling  of  interface  requirements  of  a 
system  as  a  member  of  an  SoS/FoS 

Verification 

97 

Describe  the  purpose,  inputs,  and 
outputs  of  the  Verification  process 

Apply  the  Verification  process  during  each 
phase  of  an  acquisition  program  in  order  to 
ensure  the  system  elements  meet  or  exceed 
their  requirements 

Plan  design  solution  verification  for  a 
system  in  an  acquisition  program 
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98 

Explain  how  technical  and  operational 
performance  is  verified  against 
established  baseline  in  accordance  with 
policy  and  legislation  from  component  to 
system  throughout  the  life  cycle 

Demonstrate  that  the  proposed  design 
solution  meets  requirements  and  is 
achievable  within  existing  programmatic 
constraints 

Create  a  verification  requirement  for  each 
performance  requirement  “shall  statement” 
and  incorporate  them  into  a  narrative 
section  four  in  the  specification 

99 

Identify  deficiencies  that  keep  systems 
from  meeting  requirements 

Distinguish  between  the  systems  verification 
planning  process  and  mandatory  test  and 
evaluation  planning  requirements 

100 

Explain  how  verification  of  interface 
requirements  are  captured  in 
development  and  test  plans 

Validation 

101 

Describe  the  purpose,  inputs  and  outputs 
of  the  Validation  process 

Apply  the  Validation  process  during  each 
phase  of  an  acquisition  program 

Plan,  develop,  and  evaluate  tests 
throughout  a  system’s  acquisition  life 
cycle,  e.g.,  verification,  validation, 
accreditation  (VV&A),  independent 
verification  and  validation  (IV&V),  metrics, 
technical  performance  measurement 
(TPM),  and  open  systems  (conformance 
testing) 

102 

Demonstrate  requirements,  logical  solution, 
and  design  solution  validation 

Transition 

103 

Describe  the  purpose,  inputs  and  outputs 
of  the  Transition  process 

Apply  the  Transition  process  during  each 
phase  of  an  acquisition  program 

Plan  the  fielding  process  for  a  system  in  an 
acquisition  program 

104 

Define  transportation,  packaging, 
handling,  and  storage  requirements 

Analyze  a  deployment  system  architecture 
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Overall 

105 

Identify  the  programmatic  value  added  by 
thorough  application  of  SE  processes. 

Critique  the  programmatic  value  added  by 
thorough  application  of  SE  processes. 

106 

Prepare  a  SEP  that  describes  the  well 
thought  out  SE  plan  for  individual  acquisition 
programs. 

Evaluate  a  SEP  that  describes  the  well 
thought  out  SE  plan  for  individual 
acquisition  programs. 

107 

Conduct  the  analyses  necessary  for 
preparing  the  required  statutory,  regulatory, 
and  contract  reporting  requirements  in  each 
acquisition  phase. 

Explain  the  analyses  necessary  for 
preparing  the  required  statutory, 
regulatory,  and  contract  reporting 
requirements  in  each  acquisition  phase. 

Concept  Refinement  (CR) 

108 

Identify  capability  gaps  and  potential 
materiel  solutions  which  should  be 
supported  by  a  robust  analytical  process 
incorporating  innovative  practices  -  including 
best  commercial  practices,  collaborative 
environments,  modeling  and  simulation  and 
electronic  business  solutions. 

Appraise  capability  gaps  and  potential 
materiel  solutions  which  should  be 
supported  by  a  robust  analytical  process 
incorporating  innovative  practices  - 
including  best  commercial  practices, 
collaborative  environments,  modeling  and 
simulation  and  electronic  business 
solutions. 

109 

Employ  the  systems  engineering  processes 
to  conduct  an  Analysis  of  Alternatives  for  the 
selected  concept. 

Evaluate  an  Analysis  of  Alternatives  for  a 
selected  concept  from  an  SE  perspective. 

110 

Analyze  the  selected  concept  to  arrive  at  a 
preferred  solution, 
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Ill 

Develop  the  Technology  Development 
Strategy  for  a  preferred  solution. 

112 

Describe  user  needs;  operational 
capabilities  and  environmental 
constraints. 

Interpret  user  needs;  operational  capabilities 
and  environmental  constraints. 

Formulate  user  needs;  operational 
capabilities  and  environmental  constraints. 

113 

Define  concept  performance  (and 
constraints)  definition,  and  verification 
objectives. 

Develop  concept  performance  (and 
constraints)  definition,  and  verification 
objectives. 

Create  concept  performance  (and 
constraints)  definition,  and  verification 
objectives. 

114 

Explain  how  concept  performance  is 
decomposed  into  functional  definition 
and  verification  objectives. 

Show  how  concept  performance  is 
decomposed  into  functional  definition  and 
verification  objectives. 

Appraise  the  decomposition  of  concept 
performance  into  functional  definition  and 
verification  objectives. 

115 

Explain  how  concept  functional  definition 
is  decomposed  into  concept  components 
and  assessment  objectives. 

Show  how  concept  functional  definition  is 
decomposed  into  concept  components  and 
assessment  objectives. 

Appraise  the  decomposition  of  concept 
functional  definition  into  concept 
components  and  assessment  objectives. 

116 

Identify  component  concepts,  including 
enabling/critical  technologies, 
constraints,  and  cost/risk  drivers. 

Examine  component  concepts,  including 
enabling/critical  technologies,  constraints, 
and  cost/risk  drivers. 

Assess  component  concepts,  including 
enabling/critical  technologies,  constraints, 
and  cost/risk  drivers. 

117 

Differentiate  enabling/critical 
components  versus  capabilities. 

Analyze  enabling  critical  components  versus 
capabilities. 

Assess  enabling/critical  components 
versus  capabilities. 

118 

Differentiate  the  system  concept  versus 
functional  capabilities. 

Analyze  system  concept  versus  functional 
capabilities. 

Assess  system  concept  versus  functional 
capabilities. 

119 

Describe  the  concept  and  the  system 
concept’s  performance. 

Analyze  system  concept’s  performance  for 
the  chosen  concept. 

Verify  system  concept’s  performance  for 
the  chosen  concept. 
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120 

Differentiate  concepts  versus  defined 
user  needs  and  specified  environmental 
constraints. 

Analyze  concepts  versus  defined  user  needs 
and  specified  environmental  constraints. 

Assess  concepts  versus  defined  user 
needs  and  specified  environmental 
constraints. 

121 

Recognize  that  the  technical  reviews 
normally  conducted  during  CR  include: 

TR,  which  should  provide: 

(1)  A  complete  CARD-like  document 
detailing  system  overview,  risk, 
system  operational  concept, 

(2)  An  assessment  of  the  technical 
and  cost  risks  of  the  proposed 
Program,  and 

(3)  An  independent  assessment  of  the 
Program’s  cost  estimate 

Identify  that  the  technical  reviews  normally 
conducted  during  CR  include: 

TR,  which  should  provide: 

(1)  A  complete  CARD-like  document 
detailing  system  overview,  risk,  system 
operational  concept, 

(2)  An  assessment  of  the  technical  and  cost 
risks  of  the  proposed  Program,  and 

(3)  An  independent  assessment  of  the 
Program’s  cost  estimate 

Document  that  the  technical  reviews 
normally  conducted  during  CR  include: 

TR,  which  should  provide: 

(1)  A  complete  CARD-like  document 
detailing  system  overview,  risk,  system 
operational  concept, 

(2)  An  assessment  of  the  technical  and 
cost  risks  of  the  proposed  Program,  and 

(3)  An  independent  assessment  of  the 
Program’s  cost  estimate 

122 

Recognize  that  the  technical  reviews 
normally  conducted  during  CR  include: 
ASR  which  should  provide: 

(1)  An  agreement  on  the  preferred 
system  concept(s)  to  take  forward 
into  Technology  Development, 

(2)  Software  architectural 
constraints/drivers  to  address 

Defense  Information  Infrastructure  / 
Common  Operating  Environment 
(Dll/COE)  and  system  extensibility 
requirements,  and 

(3)  An  assessment  of  the  full  system 

Identify  that  the  technical  reviews  normally 
conducted  during  CR  include: 

ASR  which  should  provide:  (1 )  An 
agreement  on  the  preferred  system 
concept(s)  to  take  forward  into  Technology 
Development, 

(2)  Software  architectural  constraints/drivers 
to  address  Defense  Information 

Infrastructure  /  Common  Operating 
Environment  (Dll/COE)  and  system 
extensibility  requirements,  and  (3)  An 
assessment  of  the  full  system  software 
concept,  and  several  other  metrics. 

Document  that  the  technical  reviews 
normally  conducted  during  CR  include: 

ASR  which  should  provide:  (1 )  An 
agreement  on  the  preferred  system 
concept(s)  to  take  forward  into  Technology 
Development, 

(2)  Software  architectural 
constraints/drivers  to  address  Defense 
Information  Infrastructure  /  Common 
Operating  Environment  (Dll/COE)  and 
system  extensibility  requirements,  and  (3) 

An  assessment  of  the  full  system  software 
concept,  and  several  other  metrics. 
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software  concept,  and  several  other 
metrics. 

123 

Identify  that  the  outputs  of  CR  include  a 
preliminary  system  specification  and  a  SEP. 

Technology  Development  (TD) 

124 

Identify  that  systems  engineering  processes 
are  used  in  TD  to  develop  a  suite  of 
technologies  for  the  preferred  system 
solution,  once  each  required  attribute  is 
converted  to  a  system  performance 
specification. 

125 

Describe  user  needs;  operational 
capabilities  and  environmental 
constraints. 

Interpret  user  needs;  operational  capabilities 
and  environmental  constraints. 

Formulate  user  needs;  operational 
capabilities  and  environmental  constraints. 

126 

Define  system  performance  (and 
constraints)  specifications  and  the 
enabling/critical  technologies  verification 
plan. 

Develop  system  performance  (and 
constraints)  specifications  and 
enabling/critical  technologies  verification 
plan. 

Assess  the  system  performance  (and 
constraints)  specifications  for  an 
enabling/critical  technologies  verification 
plan. 

127 

Explain  the  functional  definitions  for 
enabling/critical  technologies  and 
associated  verification  plan. 

Show  the  functional  definitions  for 
enabling/critical  technologies  and  associated 
verification  plan. 

Appraise  the  functional  definitions  for 
enabling/critical  technologies  and 
associated  verification  plan. 

128 

Explain  how  functional  definitions  are 
decomposed  into  a  critical  component 
definition  and  technology  verification 
plan. 

Show  how  functional  definitions  are 
decomposed  into  a  critical  component 
definition  and  technology  verification  plan. 

Appraise  the  decomposition  of  functional 
definitions  into  a  critical  component 
definition  and  technology  verification  plan. 
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129 

Identify  system  concepts,  including 
enabling/critical  technologies; 

Formulate  system  concepts,  including 
enabling/critical  technologies; 

Assess  system  concepts,  including 
enabling/critical  technologies; 

130 

Describe  changes  to  constraints  and 
cost/risk  drivers. 

Evaluate  changes  to  constraints  and 
cost/risk  drivers. 

Assess  the  impact  of  changes  to 
constraints  and  cost/risk  drivers. 

131 

Differentiate  enabling/critical  technology 
components  versus  plan 

Evaluate  the  use  of  enabling/critical 
technology  components  versus  plan 

Compare  the  use  of  enabling/critical 
technology  components  versus  plan 

132 

Describe  system  functionality  versus 
plan. 

Illustrate  the  system  functionality  versus 
plan. 

Contrast  system  functionality  versus  plan. 

133 

Differentiate  the  integrated  system 
versus  the  performance  specification. 

Illustrate/model  the  integrated  system  versus 
the  performance  specification. 

Assess  the  integrated  system  versus  the 
performance  specification. 

134 

Explain  what  the  system  concepts  and 
technology  maturity  is  versus  defined 
user  needs 

Point  out  the  system  concepts  and 
technology  maturity  versus  defined  user 
needs 

Validate  the  system  concepts  and 
technology  maturity  versus  defined  user 
needs 

135 

Identify  that  the  TRs  normally  conducted 
during  TD  include  SRR  (which  should 
provide  an  approved  system  performance 
specification,  and  several  other  metrics),  IBR 
(which  should  establish  the  Performance 
Management  Baseline,  and  assessment  of 
risk  within  it),  and  a  Technology  Readiness 
Assessment  (an  evaluation  of  system 
technology  maturity) 

136 

Identify  that  the  outputs  of  TD  include  a 
system  performance  specification  and  a 

TEMP. 
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System  Development  and  Demonstration 

137 

Recognize  that  in  SDD,  the  program  and 
the  system  architecture  are  defined 
based  upon  the  selection  and  integration 
of  the  mature  technology  suite 
accomplished  during  CR  and  TD, 

Identify  that  in  SDD,  the  program  and  the 
system  architecture  are  defined  based  upon 
the  selection  and  integration  of  the  mature 
technology  suite  accomplished  during  CR 
and  TD, 

Document  that  in  SDD,  the  program  and 
the  system  architecture  are  defined  based 
upon  the  selection  and  integration  of  the 
mature  technology  suite  accomplished 
during  CR  and  TD, 

138 

Recognize  that  in  SDD  that  system 
design  requirements  are  allocated  down 
to  the  major  subsystem  level,  and  are 
refined  as  a  result  of  developmental  and 
operational  tests,  and  iterative  systems 
engineering  analyses. 

Identify  that  in  SDD  that  system  design 
requirements  are  allocated  down  to  the 
major  subsystem  level,  and  are  refined  as  a 
result  of  developmental  and  operational 
tests,  and  iterative  systems  engineering 
analyses. 

Document  that  in  SDD  that  system  design 
requirements  are  allocated  down  to  the 
major  subsystem  level,  and  are  refined  as 
a  result  of  developmental  and  operational 
tests,  and  iterative  systems  engineering 
analyses. 

139 

Identify  that  in  SDD,  the  support  concept  and 
strategy  are  refined 

140 

Describe  user  needs,  system 
performance  specifications  and 
environmental  constraints, 

Interpret  user  needs,  system  performance 
specifications  and  environmental  constraints, 

Formulate  user  needs  system 
performance  specifications  and 
environmental  constraints, 

141 

Define  system  functional  specifications 
and  system  verification  plan, 

Develop  system  functional  specifications  and 
system  verification  plan, 

Assess  the  system  functional 
specifications  and  system  verification  plan, 

142 

Trace  the  functional  performance 
specifications  into  Configuration  Item 
(Cl)  functional  (“design  to”)  specifications 
and  Cl  verification  plan 

Convert  functional  performance 
specifications  into  Configuration  Item  (Cl) 
functional  (“design  to”)  specifications  and  Cl 
verification  plan 

Change  functional  performance 
specifications  into  Configuration  Item  (Cl) 
functional  (“design  to”)  specifications  and 

Cl  verification  plan 

143 

Trace  the  Cl  functional  specifications  into 
product  (“build  to”)  documentation  and 
inspection  plan 

Convert  Cl  functional  specifications  into 
product  (“build  to”)  documentation  and 
inspection  plan 

Change  Cl  functional  specifications  into 
product  (“build  to”)  documentation  and 
inspection  plan 
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144 

Describe  how  to  fabricate,  assemble,  or 
code  to  “build  to”  documentation, 

Interpret  what  it  means  to  fabricate, 
assemble,  code  to  “build  to”  documentation, 

Assess  how  to  fabricate,  assemble,  code 
to  “build  to”  documentation, 

145 

Explain  verification  of  Individual  CIs 

Formulate  verification  of  individual  CIs 

Assess  verification  of  Individual  CIs 

146 

Explain  value  of  Developmental  Test  and 
Evaluation  (DT&E),  Integrated  DT&E, 

Live  Fire  Test  and  Evaluation  (LFT&E) 
and  Early  Operational  Assessments 
(EOAs)  verify  performance  compliance 
to  specifications, 

Interpret  how  Developmental  Test  and 
Evaluation  (DT&E),  Integrated  DT&E,  Live 

Fire  Test  and  Evaluation  (LFT&E)  and  Early 
Operational  Assessments  (EOAs)  verify 
performance  compliance  to  specifications 

Assess  how  Developmental  Test  and 
Evaluation  (DT&E),  Integrated  DT&E,  Live 
Fire  Test  and  Evaluation  (LFT&E)  and 

Early  Operational  Assessments  (EOAs) 
verify  performance  compliance  to 
specifications 

147 

Explain  the  value  of  System  DT&E, 

LFT&E  and  OAs,  to  verify  system 
functionality  and  constraints  compliance 
to  specifications 

Interpret  how  System  DT&E,  LFT&E  and 

OAs,  verify  system  functionality  and 
constraints  compliance  to  specifications 

Assess  how  System  DT&E,  LFT&E  and 

OAs,  verify  system  functionality  and 
constraints  compliance  to  specifications 

148 

Explain  the  value  of  combined 
DT&E/OT&E/LFT&E,  to  demonstrate  the 
system  to  specified  user  needs  and 
environmental  constraints 

Interpret  how  combined 

DT&E/OT&E/LFT&E,  to  demonstrate  the 
system  to  specified  user  needs  and 
environmental  constraints 

Assess  how  combined 

DT&E/OT&E/LFT&E,  to  demonstrate  the 
system  to  specified  user  needs  and 
environmental  constraints 
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149 

Identify  that  the  TRs  normally  conducted 
during  SDD  include  IBR  (for  EVM  contracts), 
SRR  (which  should  provide  an  approved 
system  performance  specification,  and 
several  other  metrics),  SFR  (which  should 
provide  an  established  system  functional 
baseline,  and  several  other  metrics),  PDR 
(which  should  provide  an  established  system 
allocated  baseline,  and  several  other 
metrics),  CDR  (which  should  provide  an 
established  system  product  baseline,  and 
several  other  metrics),  TRR  (which  should 
ensure  that  the  subsystem  or  system  under 
review  is  ready  to  proceed  into  formal  test), 
SVR  (which  should  establish  and  verify  final 
product  performance  and  provide  inputs  to 
the  creation  of  the  CPD),  PRR  (which  should 
assess  the  manufacturing  and  quality  risk  as 
the  program  proceeds  into  production),  (TRA 
-  an  evaluation  of  system  technology 
maturity),  and  perhaps  OTRR  (which 
ensures  that  the  “production  configuration” 
system  can  proceed  into  Operational  Test 
and  Evaluation  (OT&E)  with  a  high 
probability  the  system  will  successfully 
complete  operational  testing). 

150 

Identify  that  the  outputs  of  SDD  include  an 
updated  TEMP,  SEP,  risk  assessment,  TRA, 
and  inputs  for  the  CPD. 
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LO# 

Level  1 

Level  II 

Level  III 

SE  by  PHASES  * 

*  NOTE:  Levels  1  and  III  will  be  reworked  by  DAU  to  reflect  the  most  appropriate  learning  outcomes  for  these  levels. 

Production  and  Deployment 

151 

Identify  that  the  T&E  process  frequently 
reveals  issues  that  require  improvements  or 
redesign,  and  that  the  initial  manufacturing 
process  may  reveal  issues  that  were  not 
anticipated,  or  it  may  be  discovered  that 
changing  the  product  somewhat  may  provide 
enhancements  in  the  manufacturing 
process. 

152 

Identify  deficiencies  to  determine 
corrective  actions 

Analyze  deficiencies  to  determine  corrective 
actions 

Assess  deficiencies  to  determine 
corrective  actions 

153 

Explain  how  to  modify  configuration 
(hardware,  software,  and  specifications) 
as  necessary  to  correct  deficiencies 

Interpret  how  to  modify  configuration 
(hardware,  software,  and  specifications)  as 
necessary  to  correct  deficiencies 

Assess  modifications  to  the  configuration 
(hardware,  software,  and  specifications)  as 
necessary  to  correct  deficiencies 

154 

Explain  what  is  meant  by  “Verify  and 
Validate”  the  production  configuration. 

Interpret  how  to  “Verify  and  Validate” 
production  configuration 

Assess  how  to  “Verify  and  Validate” 
production  configuration 
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LO# 

Level  1 

Level  II 

Level  III 

SE  by  PHASES  * 

*  NOTE:  Levels  1  and  III  will  be  reworked  by  DAU  to  reflect  the  most  appropriate  learning  outcomes  for  these  levels. 

155 

Identify  that  the  technical  reviews  normally 
conducted  during  P&D  include  IBR  (for  EVM 
contracts),  OTRR  (which  ensures  that  the 
“production  configuration”  system  can 
proceed  into  Operational  Test  and 

Evaluation  (OT&E)  with  a  high  probability  the 
system  will  successfully  complete 
operational  testing),  and  PCA  (confirms  that 
the  manufacturing  processes,  quality  control 
system,  measurement  and  test  equipment, 
and  training  are  adequately  planned,  tracked 
and  controlled). 

Operations  and  Support 

156 

Identify  that  SE  is  involved  in  in-service 
reviews  and  trade  studies  and  decisions 
regarding  modifications,  upgrades,  and 
future  increments  of  the  system.  This  may 
include  system  upgrades  for  interoperability 
or  technology  improvements,  parts  or 
manufacturing  obsolescence,  aging  aircraft 
(or  system)  issues,  premature  failures, 
changes  in  fuel  or  lubricants,  Joint  or  service 
commonality,  etc. 
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LO# 

Level  1 

Level  II 

Level  III 

SE  by  PHASES  * 

*  NOTE:  Levels  1  and  III  will  be  reworked  by  DAU  to  reflect  the  most  appropriate  learning  outcomes  for  these  levels. 

157 

Describe  the  performance  of  the 
following  SE  activities:  monitor  and 
collect  all  service  use  data;  analyze  data 
to  determine  root  cause  of  problem; 
determine  the  risk/hazard  severity  of  the 
system;  develop  corrective  action; 
integrate  and  test  corrective  action; 
assess  risk  of  improvements;  implement 
and  field 

Demonstrate  the  performance  of  the 
following  SE  activities:  monitor  and  collect  all 
service  use  data;  analyze  data  to  determine 
root  cause  of  problem;  determine  the 
risk/hazard  severity  of  the  system;  develop 
corrective  action;  integrate  and  test 
corrective  action;  assess  risk  of 
improvements;  implement  and  field 

158 

Identify  that  the  technical  review  normally 
conducted  during  O&S  is  the  ISR  (which 
should  provide  an  overall  System  Hazard 

Risk  Assessment,  and  an  operational 
readiness  assessment  in  terms  of  system 
problems  (hardware,  software,  and 
production  discrepancies)). 

Disposal 

159 

Identify  compliance  with  applicable 
environmental  regulations. 
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Level  1 

Level  II 

Level  III 

SE  TOOLS  AND  TECHNIQUES 

Systems  Engineering  Plans 

160 

Identify  the  proper  points  within  the 
program’s  lifecycle  to  generate  and 
update  a  SEP 

Appraise  the  SEP  process  (how  it 
describes  the  program’s  overall  technical 
approach:  processes,  resources,  metrics, 
and  technical  reviews.) 

161 

Identify  the  critical  contents  of  a  SEP: 
government  and  contractor  SE  processes, 
technical  baseline  approach,  and  the 
relationship  and  integration  of  SE  within  the 
program  management  organization  and  to 
other  program  control  tools  such  as  the  IMP, 
IMS,  EVMS,  and  TPMs. 

Critique  SEPs  for  proper  content  and 
attributes. 

Work  Breakdown  Structure 

162 

Recognize  how  to  translate  system 
definition  into  a  Work  Breakdown 
Structure. 

Explain  the  system  products  and  services 
identified  in  the  WBS. 

Evaluate  the  WBS  and  the  Systems 
Engineering  activities  that  should  be 
covered  in  the  WBS. 

Value  Engineering 

163 

Illustrate  how  value  engineering 
methodologies  support  the  systems 
engineering  process. 

Evaluate  the  value  engineering 
methodologies  in  relationship  to  systems 
engineering 

Technical  Performance  Measurement 

164 

Describe  how  TPM  can  be  used  as  a 

SE  metric  to  monitor  contractor 
progress 

Distinguish  the  role  of  TPM  in  the  systems 
engineering  process. 

Evaluate  TPM  as  a  SE  metric  to  monitor 
contractor  progress  in  systems 
engineering  planning. 
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SE  TOOLS  AND  TECHNIQUES 

Trade  Studies 

165 

Describe  how  TPM  can  be  used  as  a 

SE  metric  to  monitor  contractor 
progress 

Describe  the  studies  planned  to  make  trade¬ 
offs  among  stated  requirements;  design; 
project  schedule;  functional  and  performance 
requirements;  function;  task;  and  decision 
allocation  between  human,  software,  and 
hardware  and  life  cycle/design  to  cost 

Analyze  trade-offs  among  stated 
requirements;  design;  project  schedule; 
functional  and  performance  requirements; 
function;  task  and  decision  allocation 
between  human,  software,  and  hardware 
and  life  cycle/design  to  cost 

166 

Identify  the  methods  and  tools  for 
trade-off  analyses,  systems  and  cost 
effectiveness  analyses,  and  risk 
management. 

Describes  the  use  of  criteria  for  decision¬ 
making  and  trade-off  of  alternative  design 
solutions. 

Defend  the  use  of  criteria  for  decision¬ 
making  and  trade-off  of  alternative  design 
solutions. 

167 

Describe  the  MOEs,  how  they  interrelate, 
and  criteria  for  the  selection  of  measures  of 
performance  (MOPs)  to  support  the  evolving 
definition  and  verification  of  the  system. 

Analyze  how  analytical  results  will  be 
integrated. 

Modeling  and  Simulation 

168 

Examine  how  Modeling  and  Simulation  can 
aid  the  SE  process  particularly  in  the  pre¬ 
systems  acquisition  phases  and  during  the 
SDD  phase. 

Explain  how  Modeling  and  Simulation  can 
aid  the  SE  process. 

Failure  Modes,  Effects,  and  Criticality  Analysis  (FMECA) 

169 

Recognize  FMECA  as  a  tool  which 
analyzes  potential  failures  to  determine 
the  effect  on  the  entire  system. 

Apply  FMECA  and  demonstrate  its  effect  on 
cost,  schedule,  and  performance  objectives. 

Specify  appropriate  use  and  tailored 
modification  of  FMECA  as  part  of  the 
systems  engineering  process  to  control 
cost,  schedule,  and  performance. 
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LO  #  Level  I  Level  II  Level  III 

SE  TOOLS  AND  TECHNIQUES 

Requirements  Traceability  Matrix  (RTM) 

170  Apply  RTM  to  a  specific  systems  engineering  Specify  appropriate  use  and  tailored 

process  and  ensure  through  analysis  that  all  modification  of  the  RTM  as  a  part  of  the 
requirements  parent/child  can  be  tracked,  systems  engineering  process  to  control 
traced,  identified,  documented  and  verified.  performance  objectives. 

Technical  Data  Packages 

171  Examine  the  purpose  and  timing  of  technical  Explain  the  purpose  of  the  technical 

data  packages.  data  package  and  other  systems  specific 

information  over  the  system’s  lifecycle. 

Specifications 

172  Distinguish  how  the  systems  engineering  Explain  the  systems  engineering  activities 

process  contributes  to  development  of  a  that  should  be  included  in  the  system 

system  specification.  specification. 

Earned  Value  Management 

173  Describe  the  principles  of  EV.  Calculate  the  earned  value  of  a  program  Evaluate  EV  data  and  make 

during  systems  development.  recommendations  to  programmatic 

decisions  makers  based  on  the  results  of 
the  data  analysis. 

IMP/IMS 

174  Describe  the  purpose  of  IMP/IMS  and  Modify  an  IMP/IMS  in  the  technical  Analyze  an  IMP/IMS  for  completeness  as 

how  they  relate  to  the  overall  technical  scheduling  and  management  of  a  program  a  tool  for  technical  scheduling  and 

management  framework  management  of  project 
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Level  1 

Level  II 
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SE  TOOLS  AND  TECHNIQUES 

Technical  Reviews 

175 

Describe  the  purpose  of  technical 
reviews  and  how  they  relate  to  the 
overall  technical  management 
framework. 

Distinguish  between  the  different  technical 
reviews  and  show  how  the  reviews  map 
against  the  technical  products  required  from 
each  acquisition  phase  and  technical  phase 
of  effort. 

Distinguish  between  event-driven  and 
schedule-driven  technical  review  timing. 

176 

Identify  the  characteristics  that  make  a 
review  “integrated”  and  the  role  of 
independent  subject  matter  experts. 

Recognize  the  linkage  of  the  technical 
reviews  with  the  program  technical  baseline, 
the  attributes  of  the  individual  technical 
reviews,  and  the  span  of  review  criteria. 

Evaluate  a  proposed  technical  review 
plan,  including  the  entry  criteria,  exit 
criteria,  appropriateness  of  the  proposed 
Chair,  the  risk  assessment  criteria,  review 
elements,  and  participants. 

177 

Recognize  the  different  types  of 
technical  reviews  and  how  they  are 
mapped  against  system  technical 
maturity. 

Formulate  review  checklists  tailored  to  each 
review. 

Assess  the  appropriateness  of  technical 
reviews  in  a  program  as  proposed  by  the 
technical  authority  in  terms  of  proper 
timing  and  effective  conduct. 

178 

Describe  the  role  of  technical  authority 
in  technical  reviews. 

Formulate  risk  assessment  criteria  linked  to 
each  review  covering  the  full  span  of 
technical  risk  issues  applicable  to  each 
review. 

Compose  valid  entry/exit  criteria  for  a 
sample  set  of  reviews 

179 

Distinguish  between  reviews  and  identify 
entry  and  exit  criteria  for  each  type  of 
technical  review. 

Assess  the  readiness  of  a  program  to 
conduct  a  given  review. 

180 

Plan  a  CDR  together  with  who  should  chair, 
who  should  participate  from  the  Contractor, 
Gov’t,  who  should  chair  and  what  subject 
matter  experts  should  attend. 

Apply  entry  criteria  best  practice  and 
determine  whether  a  review  should  be 
conducted  (against  a  case-study  of  an 
example  program). 

M-33 


SPRDE/SE  CAREER  PATH  LEARNING  OUTCOMES  BY  CATEGORIES  AND  TOPICS 

LO# 

Level  1 

Level  II 

Level  III 

SE  TOOLS  AND  TECHNIQUES 

181 

Identify  how  effective  technical  reviews  can 
help  programs  meet  their  cost,  schedule,  and 
performance  objectives. 

Explain  the  implications  of  premature 
conduct  of  the  review  on  the  program. 
Formulate  risks  associated  with  in 
adequate  3rd  party  subject  matter  expert 
participation. 

Safety  Analysis 

182 

Recognize  that  Safety  Analysis  as 
implemented  by  Mil-Std-882D  provides 
a  tool  for  categorizing  safety  hazards 
and  risks  in  standard  universally 
understood  terms  and  for  identifying 
and  tracking  mitigating  corrections.  The 
results  of  this  analysis  allow  for  control 
of  a  program’s  cost,  schedule,  and 
performance  objectives 

Apply  Safety  Analysis  to  a  specific  system 
engineering  process  to  ensure  program 
performance  cost,  schedule,  and 
performance  objectives  are  met  as  well  as 
system  safety  hazards  and  risks 

Specify  appropriate  use  and  tailored 
modification  of  Safety  Analysis  as  part  of 
the  system  engineering  process  to  control 
program  cost,  schedule,  and  performance 
objectives  as  well  as  system  safety 
hazards  and  risks 
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LO# 

Level  1 

Level  II 

Level  III 

DESIGN  CONSIDERATIONS  * 

*  NOTE:  “Design  Considerations”  as  listed  in  Chapter  4  (Systems  Engineering)  of  the  Acquisition  Guidebook.  Each  design  consideration  should 
be  covered  to  the  extent  necessary  to  fully  cover  each  addressed  in  Chapter  4  content  and  Chapter  4  references.  (Topics  included  under 
Design  Considerations  are  Open  Systems  Design;  Modular  Open  Systems  Architecture;  Architectures  and  Interoperability;  Standardization; 
Software;  COTS  and  NDI;  Supportability;  Manufacturing  Capability  and  Producibility;  Quality  Assurance;  Reliability,  Availability,  and 
Maintainability;  Human  Systems  Integration;  ESOH;  Survivability;  Corrosion  Prevention  and  Control;  Disposal  and  Demilitarization;  Anti-Tamper; 
Information  Assurance;  Accessibility;  System  Security;  Test  and  Evaluation;  and  Sustainment. 

183 

Identify  the  full  set  of  design  considerations 
that  should  be  addressed  as  part  of  the  overall 
systems  engineering  effort 

184 

Recognize  which  of  these  considerations  are 
statutory,  which  are  regulatory,  and  which 
relate  to  operational  effectiveness  and 
suitability 

185 

Identify  waiver  processes  that  apply  to 
statutory  and/or  regulatory  requirements 

186 

Identify  design  consideration  sources 
(applicable  source  documentation/guidance, 
cognizant  organizations/offices)  pertaining  to 
these  categories 

187 

Identify  the  full  set  of  potential  system 
constraints  that  need  to  be  addressed  and 
tracked  together  with  other  design 
considerations 
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Level  1 

Level  II 
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DESIGN  CONSIDERATIONS  * 

188 

Explain  how  tradeoff  analyses  are  key  to 
understanding  the  system  design  implications 
across  design  considerations  and  constraints, 
and  how  this  is  an  elemental  part  of  the  design 
process 

189 

Explain  how  design  considerations  are  tracked 
to,  and  managed  across,  the  system  technical 
baseline.  Show  how  system  constraints  should 
be  addressed  as  part  of  the  technical  baseline 
definition 

190 

Show  how  these  design  considerations  are 
addressed  in  each  acquisition  phase  and  are 
captured  in  technical  baseline  products  unique 
to  each  phase 

191 

Explain  how  the  technical  reviews  address  and 
establish  incorporation  of  these  design 
considerations  and  constraints,  and 

incorporation  of  agreements  as  to  tradeoffs 
across  these  considerations  and  constraints 
by  all  stakeholders 

192 

Explain  the  role  of  stakeholders  in  managing 
expectations  at  each  technical  review 

193 

Explain  the  role  of  test  and  evaluation  in 
evaluating  system  compliance  in  meeting 
design  consideration  expectations  as 
documented  in  the  technical  baselines 

Appendix  N 

Education  and  Training  Review  “Tiers” 


First  Tier  DAU  Courses  for  SE  Modification 


ACQ  Courses  * 

ACQ  101 

Fundamentals  of  Systems  Acquisition  Management 

ACQ  201  A&B 

Intermediate  Systems  Acquisition 

ACQ  401 

Senior  Acquisition  Course 

ACQ  402 

Executive  Management  Course 

ACQ  403 

Defense  Acquisition  Executive  Overview  Workshop 

ACQ  404 

Systems  Acquisition  Management  Course  for  General/Flag  Officers 
ACQ  405  Executive  Refresher  Courses 

PMT  Courses  * 

PMT  250 

Program  Management  Tools 

PMT  401 

The  Program  Manager’s  Course 

PMT  402 

Executive  Program  Manager’s  Course 

SAM  Courses 

SAM  301 

Advanced  Software  Acquisition  Management 

SYS  Courses 

SYS  101  (New) 

Basic  Systems  Planning,  Research,  Development  and  Engineering 

SYS  201  A&B 

Intermediate  SPRDE 

SYS  301 

Advanced  SPRDE 

TST  Courses 

TST  301 

Advanced  Test  and  Evaluation 

Continuous  Learning  Courses: 

Technical  Reviews  (New  for  2004) 

System  Safety  Hazard  Analysis  (New  for  2004) 

*  Note:  level  4  courses  could  likely  use  the  same  SE  module,  and  as  an  expedient  could  be 
covered  by  a  guest  lecturer 

Second  Tier  DAU  Courses  for  SE  Modification 


LOG  Courses 

LOG  101 

Acquisition  Logistics  Fundamentals 

LOG  201  A&B 

Intermediate  Acquisition  Logistics 

LOG  203 

Reliability  and  Maintainability 

LOG  204 

Configuration  Management 

LOG  235  A&B 

Performance  Based  Logistics 

LOG  304 

Executive  Life  Cycle  Logistics  Management 

PMT  Courses 

PMT  352  A&B 

Program  Management  Office  Course 

PMT  403 

Program  Manager’s  Skills 

N-l 


SAM  Courses 

SAM  101 

Basic  Software  Acquisition  Management 

SAM  202 

Intermediate  Software  Acquisition  Management 

TST  Courses 

TST  101 

Introductions  to  Acquisition  Workforce  Test  and  Evaluation 

TST  202 

Intermediate  Test  and  Evaluation 

Others 

Selected  BCF  (e.g.,  Cost,  EVM,  Risk  Analyses  Courses) 

Selected  CON  Courses 

All  the  PQM  Courses 

STM  Courses 

STM  302 

Systems  Engineering  for  S&T  Managers 

Note:  Either  the  name  of  the  course  needs  to  be  changed,  or  the  curriculum  needs  to  be 
changed  to  truly  contain  SE  addressed  with  some  rigor,  remembering  that  there  is  no 
requirement  for  ACQ  101/201  in  this  career  field. 
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PM  AND  ACQ  COURSE  SUMMARIES 


ACQ101:  Fundamentals  of  Systems  Acquisition  Management 

Course  Description:  This  course  provides  a  broad  overview  of  the  DoD  systems  acquisition  process,  covering  all 
phases  of  acquisition.  It  introduces  the  requirements  generation  and  resource  allocation  processes,  the  DoD  5000 
Series  documents  governing  the  defense  acquisition  process,  and  current  issues  in  system  acquisition  management. 
Designed  for  individuals  who  have  little  or  no  experience  in  DoD  acquisition  management,  ACQ101  has  proven  very 
useful  to  personnel  in  headquarters,  program  management,  and  functional  or  support  offices. 

Objectives:  Students  who  successfully  complete  this  course  will  be  able  to  recognize: 

-  the  fundamental  precepts  and  bases  of  defense  systems  acquisition  management; 

-  the  diverse,  interrelated,  and  changing  nature  in  the  different  disciplines  of  defense  systems  acquisition 
management;  and 

-  the  regulations  and  governing  structures  of  defense  systems  acquisition  management 

Audience:  This  course  is  designed  for  military  officers,  0-1  through  0-3,  and  DoD  civilians,  GS-5  through  GS-9. 
However,  the  course  is  open  to  all  ranks  and  grades. 

Prerequisite:  None 

Recommended:  NA 

Method  of  Delivery:  Distance  Learning 

Length:  This  is  a  nonresident,  self-paced  course  available  through  the 
Internet.  Students  must  pass  the  final  examination  within  60  calendar 
days  of  the  start  date. 

IDA  Notes:  We  have  all  the  TLOs  and  ELOs  for  this  course;  the  SE  module  TLOs  and  ELOs  are  in  file  "Acq  SE 

Comp  TLO  ELO  Summary.doc",  along  with  the  ACQ  SE  competencies.  Once  the  SPRDE/SE  performance  objectives 
are  blessed  by  the  SE  Office  and  the  SPRDE/SE  FIPT,  someone  will  have  to  do  a  crosswalk  between  them  and  the 
ACQ  SE  competencies.  Also,  the  SYS  101  course  will  affect  what  SE  competencies  remain  in  the  ACQ  101  course. 

ACQ201A:  Intermediate  Systems  Acquisition  (Part  A) 

Course  Description:  Intermediate  Systems  Acquisition,  Part  A,  uses  computer-based  training  to  prepare  mid-level 
acquisition  professionals  to  work  in  integrated  product  teams  by  understanding  systems  acquisition  principles  and 
processes.  Both  ACQ201A  and  ACQ201B  are  required  for  DAWIA  certification. 

Objectives:  Students  who  successfully  complete  this  course  will: 

-  enhance  their  knowledge  of  the  business,  technical,  and  managerial  aspects  of  acquisition; 

-  understand  and  appreciate  the  critical  role  that  each  functional  discipline  plays  in  the  acquisition  process;  and 

-  using  computer-based  training,  theoretically  participate  in  simulated  integrated  product  teams  to  develop  plans  and 
resolve  problems 

Audience:  ACQ201 A  is  for  military  officers,  0-3  and  above;  civilian,  GS-9  and  above;  and  industry  equivalents  who 
are  Level  I  certified  in  acquisition.  Students  should  have  2  to  4  years  of  acquisition  and/or  logistics  experience 

Prerequisite:  ACQ1 01  Recommended:  NA 

Method  of  Delivery:  Distance  Learning  Length:  This  is  a  nonresident,  self-paced  course  available  through  the 

Internet.  Students  must  pass  the  final  examination  within  60  calendar 
days  of  the  start  date. 

IDA  Notes:  We  have  all  the  TLOs  and  ELOs  for  this  course;  the  SE  module  TLOs  and  ELOs  are  in  file  "Acq  SE 
Comp  TLO  ELO  Summary.doc",  along  with  the  ACQ  SE  competencies.  Once  the  SPRDE/SE  performance  objectives 
are  blessed  by  the  SE  Office  and  the  SPRDE/SE  FIPT,  someone  will  have  to  do  a  crosswalk  between  them  and  the 
ACQ  SE  competencies. 

ACQ201B:  Intermediate  Systems  Acquisition  (Part  B) 

Course  Description:  Intermediate  Systems  Acquisition,  Part  B,  prepares  mid-level  acquisition  professionals  to  work 
effectively  in  integrated  product  teams  by  understanding  systems  acquisition  principles  and  processes.  Both 
ACQ201A  and  ACQ201B  are  required  for  DAWIA  certification 

Objectives:  Students  who  successfully  complete  this  course  will: 

-  enhance  and  apply  their  knowledge  of  the  business,  technical,  and  managerial  aspects  of  acquisition; 

-  understand  and  appreciate  the  critical  role  that  each  functional  discipline  plays  in  the  acquisition  process;  and 

-  effectively  participate  in  integrated  product  teams  and  apply  knowledge  gained  in  ACQ201 A  to  develop  plans  and 
resolve  problems 

Audience:  ACQ201 B  is  for  military  officers,  0-3  and  above;  civilian,  GS-9  and  above;  and  industry  equivalents  who 
are  Level  I  certified  in  acquisition.  Students  should  have  2  to  4  years  of  acquisition  and/or  logistics  experience 

Prerequisite:  ACQ201A  Recommended:  NA 

Method  of  Delivery:  Resident/On-site  Length:  5  class  days 

IDA  Notes:  See  above. 

ACQ401:  Senior  Acquisition  Course 

Course  Description:  A  preeminent  course  for  members  of  the  Acquisition  Corps,  ACQ  401  is  designed  to  prepare 
selected  military  officers  and  civilians  for  senior  leadership  and  staff  positions  throughout  the  acquisition  community. 
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Objectives:  Students  who  successfully  complete  this  course  are  awarded  a  Master's  of  Science  Degree  in  National 
Resource  Strategy. 

The  Senior  Acquisition  Course  consists  of  the  entire  10-month  Industrial  College  of  the  Armed  Forces  (ICAF) 
curriculum.  The  curriculum  is  enhanced  for  designated  acquisition  students  through  four  major  elements: 

-  the  core  curriculum, 

-  mandatory  acquisition  policy  advanced  studies, 

-  advanced  studies  electives,  and 

-  research 

Audience:  Students  are  selected  by  their  respective  Services  or  agencies.  Military  officers  are  selected  as  part  of  the 
Senior  Service  School  Selection  Process  and  designated  by  the  Directors,  Acquisition  Career  Management 

Prerequisite:  None 

Recommended:  NA 

Method  of  Delivery:  Resident 

Length:  10  months 

IDA  Notes:  All  NDU  acquisition  courses  are  related  at  least  somewhat  to  systems  engineering,  but  aren’t  directly 
systems  engineering  courses.  For  example,  acquisition  core  courses  include  the  Requirements  Generation  Process, 
Risk  Management,  and  Product  Development. 

ACQ402:  Executive  Management  Course 

Course  Description:  The  Executive  Management  Course  is  for  individuals  who  are  not  graduates  of  PMT  301;  PMT 
302;  or  PMT  352,  Parts  A  and  B.  This  3-week  course  serves  senior  managers  who  interface  with,  or  otherwise  need 
to  understand,  the  defense  systems  acquisition  process.  Participants  explore  better  ways  to  support,  guide,  and 
oversee  acquisition  programs  through  case  studies  and  examples,  faculty  discussion,  and  guest  speakers  from  the 

DoD  community  and  the  defense  industry. 

Objectives:  Students  who  successfully  complete  this  course  will  be  able  to: 

-  recognize  what  issues  are  important  in  defense  systems  acquisition  at  the  executive  level,  and 

-  understand  why  these  particular  issues  are  important  from  a  macro-perspective 

Audience:  This  course  is  open  to  military  officers  and  civilians,  0-6/GS-15,  who  are  working  in  positions  requiring  an 
understanding  and  working  knowledge  of  DoD  systems  acquisition.  Additionally,  participants  of  equivalent  rank,  from 
defense  industry,  other  Federal  agencies,  and  allied  nations,  are  admitted  on  a  space-available  basis 

Prerequisite:  None 

Recommended:  NA 

Method  of  Delivery:  Resident 

Length:  15  class  days 

IDA  Notes:  The  SE  Office  probably  needs  to  have  someone  collect  relevant  SE  case  studies  and  examples  and 
have  available  personnel  who  can  function  as  guest  speakers. 

ACQ403:  Defense  Acquisition  Executive  Overview  Workshop 

Course  Description:  This  innovative  course  provides  general/flag  officers  and  Senior  Executive  Service  (SES) 
civilians  with  an  executive-level  understanding  of  the  defense  systems  acquisition  process.  The  workshop  curriculum 
is  100  percent  tailored  to  the  specific  needs  of  the  participant,  conducted  "on  demand,"  and  delivered  in  a  one-on-one 
desk-side  forum. 

Objectives:  General/flag  officers  and  SES  civilians  who  successfully  complete  this  course  will: 

-  augment  their  knowledge  of  specific  aspects  of  defense  systems  acquisition  in  one-on-one  forum, 

-  gain  an  appreciation  of  the  entire  spectrum  of  the  defense  acquisition  process  or  a  limited  number  of  specific  areas 
within  the  process,  and 

-  experience  "just-in-time"  learning  and  apply  this  tailored  learning  directly  to  real-time  issues 

Audience:  This  workshop  is  available  to  all  DoD  general/flag  officers,  political  appointees,  congressional  staffers, 
and  SES  civilian  employees.  Membership  in  an  Acquisition  Corps  career  program  is  not  required. 

Prerequisite:  None  Recommended:  NA 

Method  of  Delivery:  Resident  Length:  Variable,  depending  upon  the  number  of  topics  to  be  addressed; 

typically  one-half  or  2  days 

IDA  Notes:  The  systems  engineering  relevancy  for  this  course  is  based  on  the  student  asking  the  right  questions.  If 
the  SE  policies  get  implemented,  such  as  E10,  then  there  should  be  a  demand  for  answers  to  questions  on  SE  in 
acquisition.  Bob  says  that  he  gets  asked  questions  all  the  time.  By  directing  people  to  the  right  DAU  course  to  answer 
SE  questions,  the  courses  will,  by  necessity,  have  to  incorporate  SE  material.  It  might  be  useful  to  find  out  if  DAU 
keeps  a  list  of  the  questions  asked,  to  find  out  what  they  have  been  in  the  past. 

ACQ404:  Systems  Acquisition  Management  Course  for  General/Flag  Officers 

Course  Description:  This  1-week  course  for  general/flag  officers  and  SES  civilians  focuses  on  understanding  the 
perspectives  of  key  government  and  defense  industry  decision  makers.  The  course  includes  discussions  of  topics 
affecting  the  defense  systems  acquisition  environment.  Participants  who  are  not  graduates  of  PMT  301,  PMT  302, 
PMT  352,  Parts  A  and  B;  or  PMT  401  will  develop  an  executive-level  understanding  of  defense  systems  acquisition 
management. 
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Objectives:  Students  who  successfully  complete  this  course  will: 

-  gain  an  executive-level  understanding  of  defense  systems  acquisition  in  terms  of  what  is  important  and  why  it  is 
important; 

-  understand  recent  legislation  and  executive  actions  affecting  acquisition; 

-  refresh  their  knowledge  of  current  DoD  acquisition  policy  and  procedural  initiatives; 

-  appreciate  the  perspectives  of  the  Congress,  defense  industry,  and  executives  of  the  Office  of  the  Secretary  of 
Defense;  and 

-  apply  lessons  learned  and  hot  topics  to  their  current  acquisition  programs 

Audience:  This  course  is  for  general/flag  officers  and  SES  civilians  who  are  working  in  positions  requiring  an 
understanding  of  DoD  systems  acquisition.  Also,  participants  of  equivalent  rank  from  defense  industry,  other  Federal 
agencies,  and  allied  nations  are  admitted  on  a  space-available  basis 

Prerequisite:  None  Recommended:  NA 

Method  of  Delivery:  Resident  Length:  5  class  days 

IDA  Notes:  Again,  once  SE  policy  is  implemented,  this  course  will  need  to  include  it.  Guest  speakers  may  also  need 
to  be  supplied. 

ACQ405:  Executive  Refresher  Course 

Course  Description:  The  Executive  Refresher  Course  provides  an  acquisition  policy,  process,  and  lessons-learned 
update.  The  class  members  examine  their  role  as  acquisition  leaders  in  a  changing  environment.  Guest  speakers 
lead  discussions  on  contemporary  management  and  leadership  topics,  such  as  reform  initiatives,  partnering  with 
industry,  contracting  tools,  resource  allocations,  downsizing,  earned  value  oversight,  performance-based  logistics, 
and  supply  chain  management. 

Objectives:  Students  who  successfully  complete  this  course  will  be  able  to: 

-  understand  acquisition  management  policies,  processes,  regulations,  and  statutes;  and 

-  develop  a  leadership  role  in  a  changing  acquisition  management  environment 

Audience:  This  course  is  open  to  members  of  all  career  fields  who  are  graduates  of  PMT  301 ,  PMT  302,  or  PMT 
352B;  in  addition,  these  graduates  must  have  (or  have  been  selected  for)  the  rank/grade  of  0-6  or  GS-15  or  the 
industry  equivalent  thereof.  Applicants  who  are  not  graduates  of  PMT  302  or  PMT  352B  but  meet  the  rank/grade 
requirement  should  attend  ACQ  402 

Prerequisite:  PMT  352B  Recommended:  NA 

Method  of  Delivery:  Resident  Length:  1 0  class  days 
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IDA  Notes:  Once  SE  policy  is  implemented,  this  course  will  have  to  include  it  as  one  of  the  contemporary 
management  and  leadership  topics.  The  SE  Office  will  probably  have  to  have  available  personnel  to  serve  as  guest 
speakers. 

PM  Levell 

Course  Description:  There  is  no  Level  I  course  for  the  PM  career  field.  The  competencies  below  feed  into  the  ACQ 
101  course,  as  do  Level  I  competencies  from  other  career  fields. 

IDA  Notes:  The  Level  I  PM  competencies  in  SE  include: 

--Explain  how  the  Work  Breakdown  Structure  (WBS),  an  output  of  the  systems  engineering  process,  breaks  work  into 
product-oriented  elements  and  work  processes  allowing  acquisition  personnel  to  manage  risk  at  levels  lower  than  the 
overall  system. 

--Understand  that  the  Systems  Engineering  Process  is  the  process  of  technical  management  in  the  defense 
environment,  and  how  it  is  used  in  translating  user  requirements  into  an  integrated,  system  design  solution. 
--Understand  the  complexity  of  software  development,  its  integral  nature  to  the  Systems  Engineering  Process  (SEP), 
and  top-level  "best  practices"  for  successful  software  development. 

PMT250:  Program  Management  Tools 

Course  Description:  The  Program  Management  Tools  course  provides  application  skills  needed  in  a  program  office 
or  as  an  Integrated  Product  Team  (IPT)  lead.  It  is  a  follow-on  to  ACQ201 B  and  is  designed  to  enhance  journeyman- 
level  skills.  It  is  required,  along  with  ACQ201B,  for  Level  II  certification  in  Program  Management  (PM)  and  also 
prepares  students  for  later  work  in  the  Level  III  Program  Management  Office  Course,  PMT  352,  Parts  A  and  B. 

Objectives:  Students  who  successfully  complete  this  course  will  be  able  to: 

-  apply  best  practices  for  establishing  effective  IPTs, 

-  develop  Work  Breakdown  Structures  (WBSs), 

-  build  program  schedules  and  apply  risk  management  principles  using  state-of-the-industry  software, 

-  apply  current  cost  estimating  processes, 

-  perform  contract  planning  and  post-award  activities,  and 

-  use  earned  value  tools  and  techniques  for  program  planning  and  control 

Audience:  Target  attendees  are  civilians,  GS-12/13,  and  military  officers,  0-3/0-4,  in  the  PM  career  field.  Lower 
grades  may  apply  if  they  have  completed  ACQ201B.  Personnel  who  were  certified  Level  II  in  PM  prior  to  1  October 
2001 ,  or  are  certified  Level  III  in  other  career  fields,  who  want  to  take  PMT  352  A  and  B,  may  obtain  credit  for  PMT 
250  by  passing  an  equivalency  exam. 

Prerequisite:  ACQ201B 


Recommended:  NA 
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Method  of  Delivery:  Distance  Learning  Length:  This  is  a  nonresident,  distance  learning  course  available  through 

the  Internet.  The  course  length  is  73  calendar  days.  Students  must 
complete  modules  1  -8  (consisting  of  about  56  hours  of  work)  within  60 
calendar  days  of  the  start  date.  Module  9  is  an  exercise-based  "virtual 
classroom"  using  a  combination  of  teleconferences  and  the  Internet  and 
requiring  24  hours  of  work  over  the  last  4  days  of  the  course.  There  is  a  9- 
day  gap  between  the  online  portion  (days  1  through  60)  and  the  virtual 
classroom  (days  70  through  73). 

IDA  Notes:  We  have  all  the  TLOs  and  ELOs  for  this  course;  an  SE  ELO  appears  as  follows  under  the  TLO  "Given  a 
scenario,  produce  program  and  contract  WBS": 

--Employ  a  systems  engineering  process  to  develop  a  hierarchical  product  description 
The  Level  II  PM  competencies  in  SE  include: 

--Apply  the  systems  engineering  process  to  transform  requirements  and  constraints  into  an  operational  system 
design. 

--Understand  the  role  of  various  systems  analysis  and  control  tools  (e.g.,  Work  Breakdown  Structure  (WBS), 
technical  performance  measurements,  trade  studies,  and  modeling  and  simulation)  in  the  systems  engineering 
process. 

--Illustrate  the  role  of  test  and  evaluation  (DT&E,  OT&E,  LFT&E)  in  the  systems  engineering  and  acquisition 
management  processes. 

--Understand  the  role  of  manufacturing  considerations  in  the  Systems  Engineering  Process  throughout  the  acquisition 
life  cycle. 

PMT352A:  Program  Management  Office  Course,  Part  A 

Course  Description:  The  Program  Management  Office  Course  (PMOC),  Part  A,  is  the  first  part  of  the  Level  III 
certification  course  in  the  Program  Management  (PM)  career  field.  It  is  a  follow-on  to  ACQ201B  and  PMT250  and  is 
designed  to  train  Level  III  leaders  in  a  program  office  by  honing  analysis,  synthesis,  and  evaluative  skills.  PMT352A 
focuses  on  key  PMO  knowledge  and  skills  not  covered  in  the  prerequisite  courses. 

Objectives:  Students  who  successfully  complete  this  course  will  be  able  to: 

-  describe  the  role  of  science  and  technology  in  supporting  the  system  acquisition  process; 

-  understand  Information  Technology  assurance  measures,  and  interoperability  considerations; 

-  describe  current  manufacturing  and  logistics  concepts  and  best  practices  such  as  lean  manufacturing  and  supply 
chain  management;  and 

-  explain  appropriate  management  and  decision-making  models  to  aid  in  addressing  various  acquisition  program 
issues  (business  and  financial;  international;  environmental,  safety  and  health,  etc.) 
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Audience:  Target  attendees  are  civilians,  GS-13/14,  and  military  officers,  0-4/0-5,  in  the  PM  career  field.  Personnel 
certified  at  Level  III  in  other  career  fields  desiring  to  take  PMOC  for  Level  III  PM  certification  must  first  complete 
PMT250 

Prerequisite:  PMT250  Recommended:  NA 

Method  of  Delivery:  Distance  Learning  Length:  This  is  a  non-resident,  self-paced  course  available  through  the 

Internet.  Students  must  pass  the  final  examination  within  120  calendar 
days  of  the  start  date. 

IDA  Notes:  We  don't  have  the  materials  for  this  course,  although  we  do  have  the  Level  III  PM  Competencies,  which 
for  SE  include: 

--Originate  tailored,  value  added,  program  documentation  (e.g.  Acquisition  Program  Baseline,  Risk  Management 
Plan,  budget  estimates,  Software  Acquisition  Plan,  Systems  Engineering  Plan),  with  application  of  commercial  best 
practices  where  appropriate. 

--Evaluate  and  manage  a  systems  engineering  process  to  translate  requirements  into  integrated  design  solutions, 
ensuring  that  solutions  both  meet  current  requirements  and  facilitate  the  incorporation  of  new  technologies  and 
capabilities  to  meet  future  needs. 

PMT352B:  Program  Management  Office  Course,  Part  B 

Course  Description:  The  Program  Management  Office  Course  (PMOC),  Part  B,  is  the  second  part  of  the  Level  III 
certification  course  in  the  Program  Management  (PM)  career  field.  PMOC  is  a  follow-on  to  ACQ201  and  PMT250. 
The  classroom  component  of  PMOC,  PMT352B,  follow  PMT352A,  which  is  the  prerequisite  distance  learning 
component  of  PMOC.  These  courses  are  designed  to  train  Level  II  qualified  students  to  be  effective  PM  Level  III 
leaders  in  a  program  office  by  honing  analysis,  synthesis,  and  evaluative  skills.  PMT352B  features  scenario-based 
practical  exercises  with  topical  themes,  such  as  interoperability,  prototyping,  and  evolutionary  acquisition. 

Objectives:  Students  who  successfully  complete  this  course  will  be  able  to: 

-  lead  and  contribute  to  effective  teams  in  a  DoD  PMO; 

-  apply  critical-thinking  and  problem-solving  skills  to  system  acquisition  problems  throughout  a  defense  systems  life 
cycle; 

-  understand,  analyze,  and  develop  solutions  to  cost,  schedule,  and  performance  issues  faced  in  defense  program 
management;  and 

-  evaluate  the  tradeoffs  in  program  decisions  in  compliance  with  DoD  5000  Series  directives 
Audience:  Target  attendees  are  civilians,  GS-13/14,  and  military  officers,  0-4/0-5,  in  the  PM  career  field. 

Prerequisite:  PMT352A  Recommended:  NA 

Method  of  Delivery:  Resident  Length:  6  weeks 
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IDA  Notes:  See  above.  Also,  the  SE  Office  needs  to  have  someone  develop  or  collect  scenario-based  practical 
exercises  so  that  this  course  can  include  the  topical  theme  of  systems  engineering. 

PMT401:  The  Program 
Manager's  Course 

Course  Description:  This  course  is  an  intense,  highly  integrated  10-week  case-study-based  learning  experience. 
Group  discussions,  distinguished  guest  practitioners,  team  projects,  exercises,  simulations,  study  groups,  and  an 
elective  program  enable  the  learner  to  customize  a  portion  of  the  course.  Time  will  be  available  to  internalize  the 
material  through  independent  study  and  informal  work  with  peers.  Course  content  will  rely  upon  challenges, 
problems,  and  dilemmas  derived  from  extensive  current  interviews  with  Program  Managers  (PMs),  Program 
Executive  Officers  (PEOs)  and  other  stakeholders.  The  dilemmas  will  be  those  that  course  graduates  can  expect  to 
confront  when  they  return  to  their  workplaces 

Objectives:  Learners  who  successfully  complete  this  course  will  be  able  to: 

-  apply  critical  thinking  when  confronted  by  problems  and  dilemmas  on  a  day-to-day  basis, 

-  lead  and  integrate  disparate  functional  groups  and  develop  a  cohesive  team  capable  of  coping  with  the  complex 
problems  common  to  Program  Management  Offices  (PMOs)  and  PEOs,  and 

-  identify  and  apply  best  business  practices  to  achieve  win-win  relationships  with  industry  partners 

Audience:  This  course  is  designed  for  specially  selected  Level  III  certified  PM  career  field  members  who  have 
demonstrated  the  potential  to  become  managers  or  deputies  of  ACAT  I  or  II  programs  or  managers  of  major  ACAT  III 
programs.  Other  specially  selected  DoD  AT&L  workforce  members  who  are  motivated  and  capable  of  becoming 
managers  of  major  integrated  product  teams,  department  or  division  heads  in  acquisition  commands,  or  senior 
managers  in  laboratories  and/or  research  and  development  centers  also  may  attend.  This  assignment-specific  course 
is  statutorily  required  for  newly  selected  PEOs,  DPEOs,  and  PMs/DPMs  of  ACAT  I,  IA,  and  II  programs.  Participants 
must  be  0-5/GS-14  or  above  with  extensive  experience  in  acquisition,  including  4  years  in,  or  in  direct  support  of,  a 
PMO 

Prerequisite:  PMT352B  for  PM  career  |  Recommended:  NA 

field;  recommended  for  other  career 
fields;  A  Secret  clearance  is  required 

Method  of  Delivery:  Resident 


Length:  10  weeks 


IDA  Notes:  We  were  told  that  the  two  400-level  courses  are  pretty  much  designed  for  the  specific  students  and  a  lot 
of  the  material  is  based  on  the  types  of  questions  they  ask.  Any  teaching  materials  DAU  may  have  for  the  course  may 
or  may  not  be  used.  Perhaps  the  challenge  is  to  get  the  PMs  to  ask  the  right  questions.  The  SE  Office  also  needs  to 
have  someone  develop  case  studies  of  good  SE  practices. 

The  Level  IV  PM  competency  for  SE  is: 

--Apply  system  engineering  processes  and  assess  the  contractor’s  system  engineering  activities  and  products 
including  logistics  support  and  manufacturing  analysis. 

PMT402:  Executive  Program  Manager's  Course 

Course  Description:  This  is  an  assignment-specific  course  designed  to  meet  the  learning  and  performance  needs  of 
newly  selected  Program  Executive  Officers  (PEOs),  Deputy  PEOs  (DPEOs),  and  ACAT  I  (ID/IC  and  IAM/IAC)  and  II 
Program  Managers  (PMs)/  Deputy  Program  Managers  (DPMs).  Skills  and  behaviors  are  developed  through  a 
concentrated  4-week  resident  period  preceded  by  approximately  60  days  of  self-assessment  and  assessment  of  your 
program  and  program  office 

Objectives:  Students  who  successfully  complete  this  course  will  be  able  to: 

-  compete  a  comprehensive  assessment  of  their  programs,  program  offices,  and  of  themselves; 

-  identify  program  and  program  office  issues; 

-fill  knowledge  needs  and  work  issues;  and 

O  -  develop  a  plan  of  action  to  better  manage  their  programs,  program  offices,  and  professional  development 

i  _ 

o  Audience:  This  assignment-specific  course  is  statutorily  required  for  newly  selected  PEOs;  DPEOs;  and  ACAT  I,  IA, 

and  II  PMs/DPMs  prior  to  assuming  the  position.  ACAT  III  PMs/DPMs,  allied  personnel,  and  industry  students  are 
eligible  to  attend  on  a  space-available  basis 

Prerequisite:  Either  PMT302  or  Recommended:  NA 

PMT352  and  PMT401 

Method  of  Delivery:  Resident  Length:  PMT402A  -  2  day  resident  workshop;  PMT402B  -  20  class  days 

IDA  Notes:  See  above. 

We  do  have  some  material  on  this  course  to  review,  such  as  the  course  overview  briefing  charts,  the  draft  master 
schedule,  the  Syllabus,  the  Assessment  Status  Briefing  Guide,  and  the  Learner's  Program  Assessment  Guide  (we 
believe  that  these  assessments  formulate  the  questions  discussed  above  during  the  2-day  resident  workshop).  These 
could  be  tailored  to  ensure  an  assessment  of  the  SE  part  of  the  PM's  program.  The  Syllabus  contains  an  interesting 
teaching  note  on  the  Systems  Perspective. 
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ACRONYMS 


AET&CD 

AFIT 

AMC 

AMSDL 

AoA 

APB 

ASR 

AT&L 

BCEFM 

CAD 

CAE 

CAIV 

CAM 

CARD 

CCB 

CDD 

CDR 

CDRL 

CDSC 

Cl 

CLM 

CM 

CMMI 


Acquisition  Education,  Training,  and  Career  Development 

Air  Force  Institute  of  Technology 

Army  Materiel  Command 

Acquisition  Management  System  Data  List 

Analysis  of  Alternatives 

Acquisition  Program  Baseline 

Alternative  System  Review 

Acquisition,  Training  and  Logistics 

Business,  Cost  Estimating,  and  Financial  Management 

Computer  Aided  Design 

Computer  Aided  Engineering 

Cost  As  an  Independent  Variable 

Computer  Aided  Manufacturing 

Cost  Analysis  Requirements  Document 

Configuration  Control  Board 

Capability  Development  Document 

Critical  Design  Review 

Contract  Data  Requirements  List 

Curriculum  Development  Support  Center 

Configuration  Item 

Continuous  Learning  Module 

Configuration  Management 

Capability  Maturity  Model  Integration 
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CMOAIPT 

Career  Management  Overarching  Integrated  Product  Team 

COTS 

Commercial  Off  the  Shelf 

CPD 

Capabilities  Production  Document 

CPR 

Cost  Performance  Reporting 

CRD 

Capstone  Requirements  Document 

CSAP 

Course  Student  Assessment  Plan 

DAB 

Defense  Acquisition  Board 

DAU 

Defense  Acquisition  University 

DAWIA 

Defense  Acquisition  Workforce  Improvement  Act 

DCMA 

Defense  Contract  Management  Agency 

DID 

Data  Item  Description 

DII  COE 

Defense  Infonnation  Infrastructure  Common  Operating 

Environment 

DoD 

Department  of  Defense 

DoDD 

Department  of  Defense  Directive 

DoDI 

Department  of  Defense  Instruction 

DOTMLPF 

Doctrine,  Organization,  Training,  Materiel,  Leadership,  Personnel, 

and  Facilities 

DS 

Defense  Systems 

DSB 

Defense  Science  Board 

DSOC 

Defense  Safety  Oversight  Council 

DT&E 

Developmental  Test  and  Evaluation 

EA 

Evolutionary  Acquisition 

ECP 

Engineering  Change  Proposal 

ET&E 

Education,  Training  and  Experience 

ELO 

Enabling  Learning  Objective 

ESOH 

Environmental,  Safety  and  Occupational  Health 
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EV 

EVM 

FA 

FAA 

FFBD 

FIPT 

FoS 

FNA 

FSA 

GAO 

IBR 

ICD 

ICWG 

IDA 

IFSP 

INCOSE 

IPPD 

IPT 

ISR 

IT 

ITR 

IV&V 

JCIDS 

JROC 

JTA 

KPP 

FFT&E 


Earned  Value 

Earned  Value  Management 
Functional  Advisor 
Functional  Area  Analysis 
Functional  Flow  Block  Diagram 
Functional  Integrated  Product  Team 
Family  of  Systems 
Functional  Needs  Analysis 
Functional  Solution  Analysis 
Government  Accounting  Office 
Integrated  Baseline  Review 

Initial  Capabilities  Document  or  Interface  Control  Document 

Interface  Control  Working  Group 

Institute  for  Defense  Analyses 

Integrated  Fogistics  Support  Plan 

International  Council  on  Systems  Engineering 

Integrated  Product  and  Process  Development 

Integrated  Product  Team 

Industrial  Security  Regulation 

Information  Technology 

Initial  Technical  Review 

Independent  Verification  and  Validation 

Joint  Capabilities  Integration  Development  System 

Joint  Requirements  Oversight  Council 

Joint  Technical  Architecture 

Key  Performance  Parameter 

Five  Fire  Test  and  Evaluation 
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LMI 


LRIP 

LSA 

LO 

MAIS 

MCSC 

MDA 

MNS 

MOP 

M&S 

MS 

NACE 

NDI 

NPS 

NS  A 

ORD 

OT&E 

OTRR 

OUSD 

PCR 

PCD 

PDR 

PM 

PPBE 

PPBES 

PRR 

PQM 


Logistics  Management  Institute 
Low  Rate  Initial  Production 
Logistics  Support  Analysis 
Learning  Outcomes 
Major  Automated  Information  System 
Marine  Corps  System  Command 
Milestone  Decision  Authority 
Mission  Needs  Statement 
Measure  of  Performance 
Modeling  and  Simulation 
Milestone 

National  Association  of  Corrosion  Engineers 

Non  Developmental  Item 

Naval  Post  Graduate  School 

National  Security  Agency 

Operational  Requirements  Document 

Operational  Test  and  Evaluation 

Operational  Test  Readiness  Review 

Office  of  the  Under  Secretary  of  Defense 

Physical  Configuration  Review 

Position  Category  Description 

Preliminary  Design  Review 

Program  Management  or  Program  Manager 

Planning,  Programming,  Budgeting,  and  Execution 

Planning,  Programming,  Budgeting,  and  Execution  System 

Production  Readiness  Review 

Production,  Quality  and  Manufacturing 
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PRR 

Production  Readiness  Review 

PSC 

Preferred  System  Concept 

QRE 

Quick  Reaction  Element 

RDT 

Rapid  Deployment  Training 

RFD 

Request  for  Deviation 

RFP 

Request  for  Proposal 

RTOC 

Reduction  of  Total  Ownership  Cost 

SAF 

Secretary  of  the  Air  Force 

SAR 

Selected  Acquisition  Report 

SDD 

System  Development  and  Demonstration 

SE 

Systems  Engineering 

SEBoK 

Systems  Engineering  Body  of  Knowledge 

SEP 

Systems  Engineering  Plan 

SFR 

System  Functional  Review 

SME 

Subject  Matter  Expert 

soo 

Statement  of  Objectives 

SoS 

System  of  Systems 

SOW 

Statement  of  Work 

Space  BAR 

Space  Broad  Area  Review 

SPRDE 

Systems  Planning,  Research,  Development,  and  Engineering 

SRR 

System  Requirements  Review 

SSP 

Signals  Intelligence  (SIGINT)  Support  Plan 

STM 

Science  and  Technology  Manager 

SVR 

System  Verification  Review 

SW 

Software 

TAA 

Technology  Area  Assessment 

TAP 

Technology  Area  Plan 

P-5 


TDS 

T&E 

TEMP 

TLO 

TOC 

TPM 

TRR 

UML 

USD 

VV&A 


Technology  Development  Strategy 

Test  and  Evaluation 

Test  and  Evaluation  Master  Plan 

Terminal  Learning  Objective 

Total  Ownership  Cost 

Technical  Performance  Measurement 

Test  Readiness  Review 

Unified  Modeling  Language 

Undersecretary  of  Defense 

Verification,  Validation  and  Accreditation 
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